Executive Summary:
In our latest webinar session, led by our global head of AI analytics and CX strategy, Yazad Karkaria, we looked into the major sector banks in India, uncovering nuanced trends and crucial insights. Let's dissect the key findings from our session.
Understanding The Engaged Customer Score
To maximise the effectiveness of our sessions, it's crucial to grasp the significance of the Engaged Customer Score (ECS). This metric plays a pivotal role in gauging user satisfaction within the realm of mobile banking. The ECS offers a comprehensive snapshot of customer feedback by tapping into data from prominent platforms like the App Store and Google Play, where users rate their experience and provide commentary. This amalgamation of scores and comments allows for a nuanced understanding of user sentiments, pinpointing both areas of praise and those necessitating enhancement.
Through meticulous analysis of ECS trends spanning the past year, we've identified standout performers and areas that warrant closer scrutiny. By delving into these insights, you can fine-tune your offerings, ensuring they align more closely with the preferences and expectations of your valued customers.
Charting the Terrain: A Closer Look at Axis, HDFC, ICICI, and Kotak
Our analysis spotlighted four significant players in the Indian banking sector: Axis Bank, HDFC Bank, ICICI Bank, and Kotak Mahindra Bank. Each bank's journey through 2023 and early 2024 unveiled distinct trajectories of performance evolution.
- Axis Bank: Emerged as the frontrunner, consistently enhancing its performance trajectory.
- HDFC Bank: Witnessed fluctuations but showcased resilience, gradually narrowing the gap with Axis Bank.
- Kotak Mahindra Bank: Encountered fluctuations, with notable setbacks but sporadic rebounds.
- ICICI Bank: Steadfast performance marred by recent subtle declines, signaling potential areas for improvement.
Unveiling ICICI's iMobile App:
Of particular interest was ICICI Bank's mobile banking app, iMobile. Despite a generally stable performance, recent months saw a subtle decline in user satisfaction. Dive deeper, and one uncovers a tale of version issues and user feedback intricacies.
Version Dynamics: Unraveling the Story
The release cycle of iMobile's latet versions unveiled a compelling narrative of user experiences. Version 17.6 initially dominated but faced stiff competition from subsequent iterations. Notably, the 18 series ushered in a wave of challenges, triggering a notable dip in user engagement and satisfaction.
With the rollout of version 18, ICICI Bank encountered unforeseen hurdles in maintaining user satisfaction levels. This release marked a significant departure from the established norms, triggering a noticeable dip in user engagement and satisfaction. Users reported encountering various issues, ranging from login difficulties to app crashes and sluggish performance.
The challenges persisted with the introduction of version 18.2, exacerbating existing concerns and introducing new pain points for users. Despite efforts to address earlier issues, users continued to report persistent reliability and stability issues, further eroding confidence in the app's performance.
Amidst the turbulence, the release of version 18.3 offered a glimmer of hope for ICICI Bank. While not a complete resolution, this iteration signaled a partial recovery from the setbacks experienced with earlier versions. Users reported a reduction in some of the previously reported issues, indicating progress in addressing underlying concerns.
Decoding User Feedback: The Ipiphany Chat GPT Integration
Our analytical prowess extends beyond numbers, delving into the qualitative realm of user feedback. Integration with Ipiphany Chat GPT enabled us to distil critical themes and pain points from user comments, enriching our understanding of app usability and reliability.
Analytical Framework: The Four Pillars of Success
Our analysis is anchored in a structured framework comprising four foundational pillars: authentication, design and UI, reliability, and features and functionality. Each pillar encapsulates critical aspects of app performance, providing a holistic lens for evaluation.
Strategic Insights and Turnkey Solutions
Armed with actionable insights, our clients gain a competitive edge in navigating the dynamic landscape of banking app performance. Our turnkey solution empowers clients to stay abreast of evolving trends, confidently driving strategic and tactical decisions.
Conclusion
Our Ipiphany solution empowers stakeholders with the tools to navigate challenges, optimise user experiences, and drive sustainable growth. Join us in unraveling the intricacies of banking app performance and charting a course towards digital excellence.
For a personalised assessment of your banking app performance or to explore benchmarking opportunities, reach out to us today. Stay tuned for more insights as we continue to decode the pulse of the banking app landscape.
Transcript
00:05 - Glenn Marvin: Hello and welcome back to another Banking App Insights Session with Touchpoint Group. We are looking at the Indian market with our Global Head of AI Analytics and CX Strategy, Yazad Karkaria. And wow, last month we looked at the performance of Kotak bank and saw how the user experience declined sharply in January due to the release of a new version. And Yazad, what are we going to be looking at this time?
00:35 - Yazad Karkaria: Thanks, Glenn. So, on my screen right now, we have the four major private sector banks in India. As we can see, we have Axis, HDFC, ICICI Bank, and Kotak.
So, as you rightly pointed out, last time we looked at Kotak because there was a really sharp drop in their performance. But what I'll do is I'll just quickly let our viewers know what is it that we are tracking. So, we've taken all the public review data that's posted by app users on Google Play and App Store, and we've taken all those reviews that have a comment attached to it because that helps us understand why certain users are happy with their banking apps, whereas others are not so happy with their banking apps.
So, with that in mind, let's look at the relative performance of these four major players in the Indian banking sector. So, to start off with, if we take a look at Axis Bank, we can see that throughout 2023 and even in 2024, so far, they have been the market leaders and their performance has only improved in the last few months. As we can see over here, they've reached the 4 mark in Feb and Jan.
Then, we can take a look at HDFC Bank over here. This is really interesting. They started off really at a lower rate compared to Axis. Their Engaged Customer Score was only close to 2.5-2.6 back in Feb 2023, and they had a major setback in September where their user experience dropped very, very sharply. However, they have turned things around quite well for themselves. In October, we can see that their performance improved sharply and now, guess what, Glenn? They are performing consistently well and they are closing the gap with Axis.
The next bank that we have is Kotak. The performance of Kotak banking app has had quite a few fluctuations where we've seen the trend of performance going down from April last year until August. Then there was some respite in September when the results improved. In November was another good month for them when user experience was at one of its highest points. But then again, we can see that in... January they had a very, very sharp drop. We had covered this last time, the key reasons why this happened. In Feb, we can see that they've only had a very partial recovery, so, they have a long way to go until they can recover completely from this sharp decline.
However, the bank that we are going to be focusing on today is ICICI Bank and their mobile banking app, which is iMobile. So, with ICICI Bank, as we can see, their performance overall was quite consistent in the range of three to 3.5, except for April 2023, when they had a sharp drop. However, we can see that in the last couple of months, Glenn, they are gradually declining. The performance is gradually declining. It's not a major or a significant drop, but it's something interesting that's going on there.
03:43 - Glenn: It is a very subtle drop, isn't it? And how can we look at this and how can it be explained?
03:48 - Yazad: Yes, sure, Glenn. it is a subtle drop, but we have a clear story to tell over here that will explain this decline.
So, what we are going to do first is we are going to take a look at the different versions of the Android app that were released recently. In this table, I'll just take a moment to explain what's going on. On the left hand side over here, I have the different versions of the app which were released. So 18.3 is the most recent version, and 17.6 is an earlier version which was there. And over here, I have different months. We have December 23, Jan 24 and Feb 24, and we are looking at which version was most dominant in which particular month.
Here, the numbers in these cells indicate the number of reviews that we received from iMobile users in this time period. As we can see, in December, 17.6 was the most dominant version, and then in January, it remained the most dominant version over here. But we have another development that took place. We had version 18, which was launched, as we can see. So, version 18 appeared sometime in January, and then it has continued to be there even in February.
Now, coming to February 24, which is the most recent month, 17.6 does continue to remain the dominant version. But we've had a series of few quick releases from ICICI Bank. We had 18.1, which really had a small base size, so, we haven't covered it over here. But then we have 18.2 and 18.3 being released in quick succession as well. So, quite a few activities have taken place in terms of the release cycle. And let's just take a really quick, high level view of how these versions have performed.
So, what I'm doing over here is I have actually clicked on version 17.6 in the month of Feb to just get a quick understanding of what are the key themes that are coming through. Here I can see a lot of 'easy to use', 'good application', 'good service' and things that are being mentioned by customers. Let's move now to version 18 and see how the discussions change.
So, here with version 18, you see a lot of red coming up where users are saying, 'it's not working', 'troubling' 'opening the app' 'the last day it worked' and things like that, 'not able to log in', etc. Let's look at 18.2 as well. Being able to log in is one of the main themes that have come up over here, along with it being a new update, update related theme, and then looking at 18.3, let's see how this changes. Okay, this is interesting. There are quite a few customers talking about 'process request'. So, what we can do is just out of curiosity, we can just dive into this and see what is it they are talking about. So, 'why am I getting this error? FS. We are unable to process your requests' and quite a few customers have spoken about this. Some of these also have numbers attached to them. 'FS 10, etc. 'FS 31', as we can see. So yes, there are some issues going on over here regarding processing customer requests in this newer version of the app, which is 18.3, as we can see.
However, right now, Glenn, we've only analysed these versions at a very high level based on the key themes coming up. But we do have a systematic structure that we use to analyse app feedback.
On my screen right now, I'll share with you what we call the Foundational Pillars. Based on our extensive research of analysing millions of user reviews every year across over 100 banks, we have arrived at this framework, which explains that in order for any banking app to perform well, it needs to do well on these four pillars, that is, Authentication, Design and UI, Reliability and Stability and Features and Functionality. These are the main pillars that we track, and within each one of these, there's a lot of detail that we also cover in our reporting and analysis.
So, with that in mind, I am now going to take you on a quick journey where we will be looking at these different versions based on their performance.
So, I'll take a minute to explain what's going on. Here, version 17.5 is the one that we saw, which was doing really well, and we can see that it had an Engaged Customer Score of 3.5 when 17.6 was launched, the Engaged Customer Score dropped marginally to 3.3. Then, when the 18 series got launched with version 18, the Engaged Customer Score dropped very sharply from 3.3 to straightaway 2.4. This further dropped to 2.3 when version 18.2 was launched, and then, some good news for ICIC Bank is with the launch of 18.3, they have managed to recover some portion of what they lost earlier.
So, what we are going to do, Glenn, over here, is we are going to understand this change in Engaged Customer Score in relation to the Foundational Pillars Analysis that we have.
Here, in this analysis, all the numbers that you see are related to pain points. So, Reliability and Stability. In version 17.5, about 23.5% of the users complained about Reliability Instability related issues. This then increased to 32.1 with the launch of version 17.6. However, this increased to 54.8 when version 18 was released. That's a really sharp increase in Reliability and Stability related issues, as we can see over here. More than 1 in 2 of your customers are giving you feedback, which is regarding Reliability and Stability related issues they have faced. It continued to remain at a similar level even with 18.2. But the good news, as I mentioned earlier, is that with 18.3 we are seeing signs of this coming down. However, they have a long way to go for these issues to come back in line with what they were before version 18 series were launched. But yes, the direction is good.
Now, looking at Authentication related issues, we have a similar story. So, earlier when we were looking at version 17.5, only about 5% users have reported Authentication related issues, but this has increased to about 28% when we are looking at version 18. And it continued to remain high even with 18.2. And even with 18.3, it's in a much higher range than what it was earlier.
Clearly, the new versions which have been launched, we are seeing a direct correlation of the Engaged Customer Score with the performance of these versions on App Reliability and Authentication. On the other hand, we can very clearly make out that ICIC Bank has not made any significant changes to the Design element of the app, or even the Features and Functionality for that matter.
We are seeing a marginal change over here, but the main feedback is coming around App Usability in terms of the app being reliable to use and being able to log into the app. Now, of course, we've seen a lot of numbers and we've seen why the Engaged Customer Score has dropped over time. Let's get into a few more details over here and try and see what's happening. If I click on 18.2, what happens is this panel opens up on the right hand side, and Glenn, recently we have added a new feature in Ipiphany, which is an integration with ChatGPT, where ChatGPT then helps us summarise the key comments which are a part of Reliability and Stability.
So, over here, what we are seeing is we are seeing the top 5 issues mentioned by customer within App Reliability and Stability. Customers are talking about Login Issues, App Crashes and Bugs, the app performing slowly, facing issues after updates and their experience with customer service over here, as we can see.
Now, that's not just it. We have a lot of additional information also that we can drill into. So, here is the dynamic word cloud that we have seen. And if I just click on this, I can actually go in and look at the real customer comments which are behind all these numbers. We started off at a really high level where we saw the decline in the score, then, we understood where that decline is coming from in terms of our text analytics framework, and now we are looking at the actual customer comments pertaining to each of these pain points. Now, one more interesting thing over here is that our analysis is not just restricted to the Foundational Pillars, it goes into a lot of detail.
If I look at the subtopics that we have across these pillars, then, there's a lot to talk about and a lot to discuss, Glenn. But today, I'll just keep it short and sharp over here.
12:57 - Glenn: Yeah, I can see so much information in one screen, and this is really just the tip of the iceberg in regards to the depth of the analyses that you can go through. So, I'm sure there are some very happy customers of ours that are using Ipiphany and getting the most out of the platform.
13:16 - Yazad: Absolutely, Glenn. So, what we've done is we've done this analysis retrospectively, but our banking clients have access to this information real time, and with our consulting support, we support the products, the product team and the insights team with the banks that we work, they can stay on top of these budding issues which come up, and we specialise in Version Transition Analysis, where we work very closely with their teams to track user experiences with new versions and how things are changing for them. Also, the beauty of this solution is that it's turnkey. We manage the data insights generation and reporting for our clients, who can then focus on driving the key findings internally at a strategic and a tactical level.
13:55 - Glenn: Yeah, that's brilliant. Great insights yet again, Yazad. I'm looking forward to seeing what you drum up next month as well. But in the meantime, if you want to have a look at your banking app performance, even compare it and benchmark it to some of your competitors, feel free to sing out. We're more than happy to tee up a demo, and we will see you next month if we don't see you online.