#NeotysPAC – How to Start Testing Mobile Performance Earlier in the Dev Cycle, by Sofia Palamarchuk

With the global health crisis accelerating the shift to digital, consumers expect online experiences to exceed face-to-face experiences, even from their mobile phones. In 2020, mobile web visits finally surpassed desktop (54% vs 46%), according to a survey by Statista. With so many consumers relying on their phones to interact with brands, mobile performance becomes paramount. Recent studies show that 80% of people have deleted a mobile app because it had too many performance issues. This abandonment rate impacts the digital transformations that many companies are going through today. 

In this blog, I want to talk about how performance impacts user experience in mobile and introduce some of the differences between web and mobile apps. Then, I’m going to give some examples of important client-side performance metrics you can measure, tools to obtain them, and how you can incorporate these tests early on in development and testing stages.

Mobile performance is the main factor that impacts user experience 

Of course, performance is not the only aspect that impacts a mobile user’s experience, but it is one of the main contributors to poor UX, since:

  • Slow pages are the number one issue that mobile users complain about, ranking even higher than site crashes.
  • Most mobile users expect pages to load at least as fast as (or faster than) they load on their PC.
  • 65 percent of users say a poor mobile experience negatively impacts their opinion of the brand.

Mobile apps vs. mobile websites

A recent report from Sensor Tower revealed that consumer spending on mobile apps and app installs grew significantly during the first half of 2020, reaching $50.1 billion worldwide across the App Store and Google Play. The same report also estimated there were 71.5 billion first-time app installs during the first half of 2020. That is an increase of 26.1 percent year-over-year, giving even more incentive for businesses to develop an app service.

Another fact to keep in mind is that native mobile apps can run up to five times faster than websites (if implemented properly :)).

So, does that mean you should develop a native app instead of having a mobile-friendly website? 

The right choice will depend on your business objectives. If your goal is to offer mobile-friendly content to a wide range of people, then a mobile-friendly website is probably the way to go. However, if you want to engage better, interact with, and communicate with your customers to drive customer loyalty, a mobile app presents itself as a better option.

Some benefits of mobile apps:

  • More interactive ways for the user to engage
  • Better personalization
  • Works offline
  • Intuitive interface: use of gestures
  • Uses mobile device features – camera, contact list, GPS, phone calls
  • Works faster than a website

Some benefits of mobile websites:

  • Accessible to everyone through a browser
  • No need to download or update
  • Cost-effective

If you do choose to go with a mobile website, you can consider using newer technologies for the benefit of performance:

  • AMP: An open source initiative originally developed by Google and Twitter. Known in its infancy as Accelerated Mobile Pages, one of the project’s goals was to help create optimized mobile web pages.
  • Progressive web apps: At their heart, progressive web apps are just web applications built and enhanced with modern APIs to deliver native-like capabilities, reliability, and installability while reaching anyone, anywhere, on any device with a single codebase. That’s why these apps are seen as “the best of both worlds.”

Client-side performance

Having looked at the differences in mobile vs. web apps, let’s talk about how to measure performance and what to measure on the client side.

While mobile and web applications have some similarities, there are many striking differences between the two. Mobile applications depend on the underlying frameworks provided by their operating systems (iOS/Android). However, web applications, which have similar functionality to desktop applications, have a client–server architecture where the client (including the user interface and client-side logic) runs in a web browser.

Due to their differences, mobile and web applications have different metrics that we can monitor to provide a holistic overview of how our apps are performing.

What should you measure for a web app?

In mobile devices, client-side processing is constrained by slower CPUs and less memory. Make sure to start measuring/checking for: 

  • Loading and rendering below-the-fold content
  • JavaScript parsing time
  • Image size
  • Caching policies

What should you measure on a mobile app?

When looking at mobile-native apps, we need to take into consideration that our app is using shared resources with other apps that are running in the device, and we should try to optimize this usage as much as possible. 

Make sure to start measuring/checking for: 

Device resource usage:

  • % CPU
  • Memory (Mb)
  • Data sent and received (Kb)
  • % Battery usage

Rendering:

  • Rendering times (ms)
  • Frame per second (fps)
  • Render lag (janks) — users can experience lags in screen drawings

Errors:

  • Exceptions
  • ANRs (“application not responding” errors) in Android 
  • Response times of actions or transactions measured from the user perspective, such as the time that it takes for a user to successfully log in to the app

Tools for testing mobile performance

When it comes to mobile web apps, there are tools for measuring their performance such as Lighthouse, from Google’s web dev tools, which can help you gather a lot of information, including optimization tips. You can also use APM tools, which show things like how to load JS resources, cache policies and more. 

For native apps, there are profiling tools like Android Studio or Instruments. These are of great help when debugging an app and looking for the root cause of performance issues.

On top of that, my startup recently launched our first free product, Apptim Desktop, focused on capturing performance data on the client side during a manual exploratory test. With Apptim Desktop, we wanted to solve the problem for devs and testers of not having enough time, expertise or tools to test for mobile performance early on and help avoid performance issues from going live to their users. 

Here are some screenshots of an Apptim Report, generated with our tool during an exploratory test:

Test mobile performance early 

As most software quality professionals would advise, the key is to start testing as soon as possible. In my opinion, this is especially important for performance teams working on mobile applications for two reasons. The first is because it gives you time to understand how the app behaves, familiarize yourself with these performance metrics and test on different devices. And secondly, it’s interesting to observe how these metrics change over time, as it will help the team identify what changes to the app can affect its performance. This also gives the team more time to take action, preventing new performance issues from reaching the end users. 

Fortunately, adding client-side performance tests does not have to involve added effort, since you can capture these key metrics at the same time as you run manual tests using Apptim.

I hope this post has inspired and empowered you to include client-side performance tests early in mobile app development. If you have any questions or want to learn more, visit the Apptim website and blog or reach out to me on twitter (@sopalamarchuk). We are always looking for testers’ and devs’ feedback so we can continue to improve our product.

If you want to know more about Sofia’s presentation, the recording is already available here.

Leave a Reply

Your email address will not be published. Required fields are marked *