Empowering NatWest’s Digital Assistant for an Improved Customer Connection
Customer Success Story:
About NatWest Group
NatWest Group is a British banking and insurance holding company. The group operates a wide variety of banking brands offering personal and business banking, private banking, investment banking, insurance and corporate finance.
Cora is NatWest Group’s customer-facing digital assistant. It provides high-quality help and support to the bank’s personal and business customers, and handles over 10 million customer conversations per year. These span a wide range of banking queries, and Cora helps customers complete some common banking tasks. The digital assistant is available across multiple channels, 24/7, so its performance is critical to ensuring unparalleled service for NatWest Bank’s Customers.
customer conversations since its introduction in 2017
NatWest has an ambitious vision for the future of Cora. The goals include:
More proactive engagement with customers
More “do it for me” journeys within Cora (such as change of address) to enhance the speed and level of service offered to customers
More “take me there” journeys via mobile deep links to resolve the customer need more quickly
Knowledge and capability growth via expansion into new areas across the bank (scaling)
Expansion into new channels including voice and smart devices
More connected data that will enable personalized journeys and offer best-in-class customer experience
As optimizing and expanding Cora’s query-recognition abilities became a key focus area, the NLU optimization team needed to more quickly identify where the NLU could be optimized in order to make improvements to the training data, as well as to have a clearer view of issues such as intent conflicts and word influence in ground truths to resolve such conflicts. The team were using benchmark testing such as K-fold and Blind Data to monitor the effect of changes, and manually implemented iterative improvements. This sometimes led to long development times for optimisations; the fact that the intuition had to identify these opportunities manually meant that upskilling new team members was a long process, too.
The Challenge
In order to address these issues, as well as to generally enhance the NLU optimization process, the Cora team looked to Cyara.
Botium has become a key tool for the NLU optimization team, reducing the time taken to identify opportunities for NLU improvement, as well as to assess the impact of changes during development. In addition, features such as confidence threshold analysis have given the team a way to easily understand the impact of these changes.
The Solution
QBox [now Cyara] has empowered my team to make a range of tangible improvements to Cora’s NLU, resulting in NLU accuracy improvements surpassing our initial goals and driving Cora towards our goal of providing unparalleled support to our customers. The range of tools on offer, as well as the support from the [Cyara] team is perfect for our needs.
- Zachary Greenan Cora NLU Product Owner, NatWest Group
Over 10.4m customer conversations took place in 2022 alone.
When going through development cycles, the Cora NLU optimization team have been able to save several hours per day, translating to hundreds of hours per year. Using Botium to supplement development has enabled the team to implement a range of tangible NLU improvements in a shorter time frame.
Outcomes
Helpful when working on improving the routing of certain queries that may have overlapping intents, or instances of incorrect routing.
Intent Conflict Visualisation:
Provides a clear view of the impact of any confidence threshold changes, ensuring the threshold works effectively in the real world and enabling the team to gain optimum levels of automation without risk.
Confidence Threshold Analysis:
Enabling the team to see the influence of words/phrases in training data so they can understand the impact of adding certain ground truths, and to react accordingly.
Word Weighting:
Cora’s training data accuracy has improved by over 8% within just 12 months – which ultimately increases the number of conversations being automated.
Improved Accuracy:
Over the last 12 months, the Cora team has adopted more Botium functions and the partnership is extending. When asked if he would recommend Cyara, Zachary Greenan (Cora NLU Product Owner) said: “Yes! The direct support from the [Cyara] team has been hugely valuable and no query has gone unanswered.”
The Journey Continues
39m
NatWest Group’s digital assistant, Cora, has completed over
customer conversations were fulfilled by Cora without requiring the intervention of a human agent…
In 2022, over
49%
more were directed to the best support channel for their query.
…and about
17%
Learn More about Botium
Learn More about Botium