Analysis of UK Banking App Performance Metrics - July 2023
The rapidly evolving environment of digital banking applications necessitates consistent analysis, particularly within the UK, given its role as a central hub for fintech innovation.
In the latest Touchpoint Group banking app insights session with CRO Glenn Marvin and head of Customer Intelligence, Tony Patrick, we uncover insights into the application performance metrics of several major banks, both challenger and traditional. This month the focus is primarily on the user experience metrics and their correlation with customer feedback.
Central to this evaluation is the 'Engaged Customer Score', a carefully constructed metric that integrates customer ratings with their qualitative comments. This offers an in-depth understanding of the performance parameters of banking apps. Notably, this metric is sensitive to nuanced alterations in user sentiment, providing an effective measure of user experience consistency across diverse banking platforms.
A case in point is the comparative analysis of Lloyds and NatWest. Lloyds demonstrates consistently positive user sentiment, in stark contrast to the recent challenges faced by NatWest. A significant decrease in the latter's engaged customer score in May and June suggests discrepancies in user experience, specifically concerning usability. The surge in feedback alludes to recent UI modifications, potentially not aligning with the user's anticipated experience.
Similarly, Monzo, a key contender in the UK's challenger banking sector, has exhibited concerns. Its score, after maintaining stability between 3.5 to 3.7, demonstrated a pronounced decline in July. Preliminary data attributes this drop to a recent application update, primarily related to design and UI elements. Terms extracted from user feedback such as "poor user interface" and difficulties in navigation suggest potential areas for optimisation. The implications of such alterations have broad ramifications, including the possibility of diminished user retention and altered brand loyalty dynamics.
Beyond scores, the analysis extends to practical application features, such as inter-account money transfers. A side-by-side evaluation of Lloyds and NatWest offers key insights. While Lloyds users predominantly report a streamlined transfer process, updates from NatWest appear to introduce unforeseen challenges to this core banking function. The discrepancies in feedback and the resultant variation in their engaged customer scores accentuate the interdependent nature of application functionality and user satisfaction.
In summary, the differentiating factor between optimal performance and suboptimal outcomes hinges on the alignment of user-centric design and feature updates. The benefits of continuous monitoring and analysis of user experience is evident and must not be ignored.
Video Transcript
Welcome back to another Touchpoint group banking app insight session. This is looking at the UK market, July 2023, and we're going to be having a little bit of a look at Monzo.
What's working and also, what's not working for them at the moment.
Then we're going to do a little bit of a deeper drive across the major banks, looking at the money transfer feature that is there, some of the best practice, some of the not so best practice, and what you can do to actually try and bridge that gap.
As usual, I have Tony Patrick with me and he's going to lead the charge on this. So, Tony, over to you to give us that broader look at the market before you jump into it.
Thank you, Glenn. And so, yeah, today, looking at these banks here in particular, just showing their trends across time recently as well.
Now, of course, with any of these, if you can't see your bank here, just give us a shout out and we can do a personal session if you like. So one of the things we'll have a look at first is, of course, what we call our Engaged Customer Score.
There's a lot happening inside here, but if we just focus on a couple here so, for example, Lloyd's, but again, the Engage Customer score is those customers giving both a score and a comment in the App Store or the Android Store. So that allows us to actually any of these movements here, we know exactly why they've happened because there's a comment attached to everything that's happening here.
Also the reason we do it that way is because these scores become then very sensitive. So we can see why even a small movement, we know why that's so one of the clear things here we're seeing across time is we were giving a good shout out to NatWest about how they'd sort of bumped up against Lloyds and were doing really well, except they did something in May and June there. And you can see the drop away we had into June and Match actually repeated itself into July. So clearly there's some issues for customers of NatWest and how they've dropped away there. So I'll have one quick look at that one slide on that in a moment.
But before we jump into that, I want to have a quick look at just some other banks and how they're going. So the likes of Starling, they dropped away for a couple of months there, but they've picked up nicely.
We can see a few things going on, like Chase is doing pretty well. They've actually picked themselves up from about a year and a half ago. They're actually doing pretty well and sort of jumping around that 4.2 level, which is pretty good.
One of the things I want to look at, though, is Monzo, so they're bumping around between sort of three and a half, 3.7, that sort of level.
But, as you can see right at the endpoint there, in July, they've dropped to their lowest level we've seen across the past year and a half. So for us to show there's something going on there, what have they done in that time period? But as I said, one of the things we'll have a look at is why NatWest has what's the issue here?
Now, we saw before this is around use. Ability issues. So we can see their usability issues have jumped up, this is around UI and things like that. So they've made a change here and customers don't like it. Right now we don't expect just something for NatWest to think about.
We don't expect this to suddenly improve their score. When you change anything, customers will have time to get used to it, but of course you have to watch out. Is this going too far? And is this score dropping too far beyond your expectations? So, I can see here usability has jumped up to about 20% of people are mentioning issues with usability and UI.
So I can see here that's an issue they need to actually address and understand in a bit more detail. What I can also see is the things like logging in issues. They've remained pretty static there's issues there, but they've remained pretty flat. The same with reliability. So reliability and logging in is all good, except the UI is causing the problem for customers.
So that's NatWest. I'll step forward now into having a look because we covered them before, so I don't want to cover them again this time around.
Let's have a look at one interesting thing happening with Monzo. Now. These scores are jumping around because the scale has actually changed. Because we're just looking at one brand at this point.
This is looking at just Monzo and we can see they're sort of moving around. The scores sort of goes 3.63.7. But we can see here in July though, it's the lowest point they've been across the past year at 3.3. So what's going on? And can they stop that from going backwards the way it has?
So, first of all, let's have a look at the app release cycle. So this is looking at from the start of May till the end of July, and these are the major releases across that time period. And what I can see here is there's one connected to those recent drops, that's 5.33 and that's the one there.
If I put my mouse over that one, I can see the average score is 3.7, whereas over the previous release the average score is 4.0. So this most recent release is having a bit more of a negative impact than previously. So let's understand why that's happened though. What are the things that are changing?
This particular view here gives us that perspective about these big pain points, what's going on? So things like reliability and stability, that hasn't necessarily it's gone up slightly in July, but it's sort of flattered around 14%. The things that have picked up though are design and UI issues. They've picked up since probably around April, but it's more emphasised though in June and July.
So those two releases we just looked at recently, 5.33, 5.33 is probably the one that's causing that maintenance of that high level design and UI issues. Features and functionality issues have also jumped up in July, but I think design and UI is the thing we need to look at because that's the thing that's causing that drop in score over those past few months.
So this is looking at Monzo, looking at the score across time and how it's curved and dropped away over those past few months. Now, one of the things about design and UI we saw it didn't actually go down or up in frequency from June, it actually remained fairly static. One of the things that we can see is that poor user interface is actually increasing in here. So let's have a look at poor user interface.
So it's only 3.4% of customers at this point, but it is increasing and it's the sort of thing you want to get on top of pretty fast. So, first of all, there's a new layout is awful, there's a 5.34. So the most recent releases coming through here, and this review is coming from right at the end of July.
So this is going to follow through into August data as well. And the layout is terrible, it's hard to navigate. So obviously people are getting used to what's going on here. So Monzo needs to keep an eye on this to make sure this isn't an ongoing problem where it's not just people getting used to it, it actually might be causing problems with extra clicks to get places.
It's hard to understand things. So need to keep on top of that. So clearly some things to focus on for Monzo.
So let's take a big switch away from that. At the moment we've seen what's happened to Monzo. What I'd like to do now is look across the market and see what's happening across various journeys or features as part of the app. So this is looking at various brands and how they perform across just looking at this year so far.
So we can see here, for example, if I look at one we're going to focus on, which is a basic thing like transferring between accounts. We can see there the average score when people talk about transferring between accounts is 4.3 for Lloyd's customers, whereas we're sitting here at about 3.5 for Natwest’s customers.
So clearly there needs to be some things there is best practice we can see there that Lloyds is doing that. So first of all, let's have a quick look at what Lloyds is up to. So what are customers talking about broadly when we talk about transfer between accounts? So what I like to look at here is the green versus red, for example.
So what I'm seeing here is that anything green is having a positive contribution towards their score. So I can see things like easy transfer money. So it's very basically it's just saying transferring money is fairly easy. If I have a quick look though at so this is transferring money.
So if I look at another one here, which is for NatWest across the past year, we can see there's a bit more red sitting in here. So moving money is difficult, transfer between accounts is difficult. And also there's an issue here with the latest update is causing an issue for customers as well, which we saw previously.
Let's have a look though. If I go back to Lloyd's and actually look at the comments and see what are the comments coming through here about what's going on? So a couple of negative things, but you can see here in general all there's a lot of green going on here.
A couple of angry customers here, they've had an issue, but generally we can see it's green. There's a lot of positive going on. Let's actually jump in and have a look at that compared to NatWest. So a similar perspective from before, we can then see how green or red this might be.
Yeah, we do get some positive things coming through. Forced to transfer money to my husband's account so he can set up the future payment. So it's an issue for that customer there. There the update is awful. So a lot of this is connected to that new update as well.
It's confusing and takes much longer to transfer money. So, again, there's clearly a best Practise here versus issues that are coming through that need to be clearly improved.