Executive Summary:
The user experience delivered by banking apps plays a pivotal role in determining an institution's market positioning within the current banking landscape. In this Touchpoint Group Banking App Insights session, Glenn Marvin and Tony Patrick undertake a granular analysis of the UK market, with a spotlight on Santander's performance in August 2023.
The methodology hinges on the 'Engaged Customer Score', a metric curated from scores and comments tendered by users. This metric proves highly sensitive, illuminating subtle movements in user satisfaction and their underlying causes.
A notable observation from the data is Lloyds Bank's consistently exemplary performance, maintaining a stable 4.5 score. This stands in contrast to the erratic scoring trajectory exhibited by other banks, including Santander.
While Santander was once a notable competitor, recent data has highlighted significant inconsistencies in their performance. Despite achieving commendable scores in the past, like in September of the previous year, the bank has experienced fluctuating scores since then. The reason? A melange of issues like app session terminations, bugs, and certain technical glitches. An incisive analysis reveals that the crux of the matter for Santander lies in its Android app version, where users frequently report being involuntarily ejected from the app.
Tony's examination doesn't end there. The deep dive includes NatWest’s performance, which, while commendable at times, witnessed a significant blip post their app update in June and July. The new user interface, although initially perceived as problematic by nearly 20% of its users, saw improved feedback over time. However, its Design and UI issues still persist, making it imperative for NatWest to direct attention towards these pain points to revert to their formerly high scores.
Broadening the lens to other banks, there's a mixed bag. Starling has been identified as a consistent top performer, whereas Monese grapples with varying performance levels, hinting at ample room for enhancement.
A noteworthy mention in this discourse is the functionality of bill payments via the app - a seemingly rudimentary feature, yet crucial for user satisfaction. Lloyds emerges as the gold standard here, with overwhelmingly positive feedback. In stark contrast, Chase faces challenges, as highlighted by a plethora of red flags in their review analysis.
This session underscores the vital necessity for banks to maintain a consistently high-quality digital experience for their users. The insights obtained from this analysis, particularly regarding Santander and NatWest, demonstrate that even minor glitches can significantly impact user satisfaction. Therefore, it's not just about excelling in the larger picture, but also refining the minutiae that can set one app apart from another in this hyper-competitive digital banking landscape.
As Glenn Marvin aptly summarises, the potential for improvement is vast, especially when institutions harness the troves of publicly available data. With the right analytical approach, institutions can indeed elevate their digital offerings to best-in-class.
Video Transcript:
Glenn Marvin (00:01.442):
Welcome to another Touchpoint Group Banking App Insights session. This session we are focusing on the UK market for the month of August 2023, and the core focus of this session with Tony Patrick is on Santander. Tony will be digging into it, looking at some of the little things that can drag a banking app's performance down, impacting the experience of its users and then look at some of the best practice models out there and those that are performing really, really well. So, you can see the difference in the impact that each one of those makes. So Tony, over to you and looking forward to seeing more of it.
Tony Patrick (00:47):
Brilliant, thank you Glenn. So as Glenn mentioned, this is the UK, so there's a few banks in here we'll go through. So, one of the first things we look at is what we call the Engaged Customer Score.This is a score given by customers. This is those who've given both a score and a comment. And what that does, it makes this very sensitive and we can see movements that are going on and also any movements we see here, we actually know why they're happening.
There's a lot of movement happening here, but again, this is a very condensed view because this is looking at the score is actually 3.5 and above. These are actually scoring reasonably highly these brands. What we can see here is that Lloyds is one of our best out of the UK. They're sitting at four and a half consistently and very solid. And that's one of the things, consistency is key here, you want that. Consistently high performance is what we're trying to achieve.
I'll look at another one in here, which is we'll look at Santander in a minute. And we can see that historically, like in September last year, they had a pretty high score, and competing strongly with the rest of the market. Although, as you can see here, there's inconsistencies, a drop in November, a drop in May, and now a drop in August, we can see here as well.
Let's have a glance at NatWest. We were amazed at how well they'd done when they were competing against Lloyds. Jumping up to the top there, great, except what happened in June and July was there was an app release with a new UI. Let's update you on that very quickly and then we can move on to have a look at Santander and see how they've moved, but briefly, let's have a look at some other banks just so we're not leaving anyone out here. What we can see is the consistent performance. Starling is sitting at the top, so you know, fairly consistent there.
The likes of Monese, they're picking up some time to gain for customers. This is a very inconsistent performance and they can do better in here as well. So let's have a look at NatWest. What we're looking at here, this is what we call our SURF analysis, looking at problems, at pain points. And what we can see here is what happened in the week commencing the 19th of June is design and UI issues that spiked up very strongly because there was a new app release, there was a new design, there was a new journey for customers and how they interact with the app. Those negative comments picked up very strongly and peaked at over 20% - looking at 1 in five customers talking about designing new issues.
What we're seeing though is, across time that's improved. And the reason behind that is generally there'll be things like NatWest would have done some work to improve some things, but in general, because they've done the pre-work, NatWest would have done to understand this is going to be best practice.
customers just need to be used to it, then it's actually going to be a better experience. Customers look to be getting used to that. You can see that pain point is dropping. Now it's only around 10% in that last week in August. This is week-by-week at the moment. What we can see here though is that is improving, but suddenly in the last week of August we've seen this jump up in reliability issues. So that had been reasonably stable, but it's jumped up at the end of August.
NatWest has got to watch out for that, as well as their design UI issues that are ongoing. We've just seen design and UI has dropped down to 10. 10% is an issue. Is that good or bad? Let's have a look. So, I'm looking at here, I'm just looking at the NatWest score for August, and it's around 10. It's actually 12.4 exactly design and UI issues. What I'm gonna look at though, the main thing we'll look at is on the right here. How does that 12.4 compare to the rest of the market?
Again, we want to get this as low as possible. They're sitting at 12.4. The only one ahead of them is 12.6. So, that is not very good. Others are getting below 5. Even though they've improved, they've still got a lot of improvement to go, and they need to keep an eye on that and also that reliability issue that suddenly jumped up in the last week of August. But again, I don't want to focus on this too much, just showing there's some issues for NatWest to uncover over the next few months until they get back to that high score they were before.
Let's have another look at, let's jump into Santander. Now, we pointed out earlier, they were doing very well in September last year, but as we can see, inconsistencies have jumped in. They dropped into October, November down to 3.75. So they're still doing reasonably well, but the consistency is why they're not up there with the likes of Lloyds and others. So we see here, May they've dropped down and August, have dropped down. So clearly some things they need to do to pick themselves up.
Let's have a quick look at August. What we're looking at here is the score across the top here is that drop in August. It's gone from 3.9 to 3.8, not much, but this is very sensitive, as I said, this measure we have. So we can see why that's happened. There's more authentication issues, and more reliability issues have happened in August compared to July. So it's gone up 3 points, about 3 points reliability, about 2 points authentication.
Now, one of the things we can look at is how does that compare to the market? So we look here on the right, we can see that I've looked at one in particular, at bugs, crashes and freezes at 3.9. 3.9 is actually above average. So it's not the worst in the market. There's actually four that are having a bigger problem. It's above the average though. So again, they need to pick up those bugs and crashes and freezes, and consistency is there, is one of their biggest problems is making sure that's consistent across time.
Again here, we can even see things like, broad technical issues at 16%. How does that compare across the market? Again, they're above average. They need to bring that way down to below 10% rather than sitting up at 16.
What I'll do now though is, I'll have a look at what's happened between September and August for Santander. So why did they drop to the level they are at the moment? So we saw, we can see things like reliability and stability issues have increased. We can see technical issues have increased. So that's interesting, they're very broad, high-level topics. What if I look down further though? I've got a few other things like app session termination issues, app bugs and crashes and freezes. So, and even things like settings and profile management.
I'll look at one of them though in particular, and just for August, which is this app session termination issue. Now, one of the things we can see on the right here is actually those actual comments coming through from that. So, customers being told the server's timed out, they lost connection, it won't let me log in, it's forgetting my login details. So, customers are being forced out of the app at unexpected times. So again, that's something to, obviously that's clearly a problem compared to the previous period in September. So first of all, so obviously it's a clear issue,
I'd like to have a look at a couple of things. I'd like to look at it in terms of, is that good or bad compared to the market. I'll just jump in here and just compare that to others in the market. Now again, we're sitting there at about 2.6%. So it's not a major thing, there's a lot of small things you need to fix to be getting to be that top level app.
So again here, what I'll do, I'll change this just to the frequency. So what's the share of the comments across here? We can see that Santander is second last. Now again, it's sitting there at 19 out of 20. What I'll do, this app login area. I'll go into that one, which is the app session termination.
They are in last place on that one. And they're more than double, they have more than double the issues compared to the second one here at Virgin Money. So clearly they need to be picking up their act in this particular area about the app just stops working. It's throwing customers out of the app. Now, one of the things you may have seen with the examples is where this is coming from. So what I can do here is look at this in terms of the operating system and I can apply that. Now I'll look at this just by a frequency count. Either way, it doesn't matter. We can see this as all coming from Android. So there's an Android issue in August around customers being chucked out of the app and they need to pick that up very quickly.
Hopefully they've seen that. If they haven't, they can see it here and they can act on that pretty fast. So let's leave Santander for the moment. And what I will do is just have a look broadly at one particular thing. So this is just looking, although Santander is here, I want to just have a look at how some of these apps perform in terms of certain journeys. And actually today I'll focus on the right here, paying bills via the app. We can see that scores are fairly high.
But there's one that is doing particularly well, which is Lloyd's and one is doing not so well, which is Chase. Let's have a look at those side-by-side. So, paying bills via the app, it should be a very basic functionality that should be working every time. So looking at Lloyd's, one of the things you see here is all the scores are very high. Actually what I'll do here, I'll actually add in here display fields. I'll actually make sure we've got things like app version, operating systems, review dates, just so we get a bit of a sense of where this is coming from, but across the board we can see this is iOS Lloyds. It's easy to access. It works very well. So very positive. Not everything's positive. Some people have an issue, but in general, it's all green.
Let's have a look and compare that to Chase. Let's look at Chase. And you see here, there's a smattering of red coming through and also blue, which are the middle level scores. So seeing there's some more issues coming through and not quite best practice the way Lloyds is. Let's have a look at though, from this perspective, which is sort of a concept cloud.
Just looking at what does it look like in terms of what are the problems and which things are over-indexing in this area. So, poor making payments, poor bill payments, poor payment feature in general. There's some positive things. Paying by cheque looks positive. But again, this is generally red, as you can see. Let's have a look at that for Lloyds. So Lloyds in here, as we expect, is green. There's some sort of average things, just making a bill payment. But in general, this is very positive.
So we can see here that it's those small things you need to be fixing. If you look back at Santander, it's small things like that app session termination issue, which needs to be looked into straight away for Android, because it's causing a current issue right now. And there's little things here, like basic stuff, like paying a bill via the app. There is best practice. There's always room to improve. So thank you, Glenn.
Glenn Marvin (11:22):
Yeah, absolutely gold there Tony and I love how we can actually get right down into and isolate even the operating system that's causing the issues here, and remember folks this is all publicly available data that we're accessing. So imagine what you could actually do when you had access to all of the information that your app users share with you. Thanks once again, looking forward to our next session.
And if you do want to speak to us around getting a demonstration or digging deeper into your own data, feel free to give us a shout.