The Best of Both Worlds – Combining Performance and Functional Mobile App Testing

With mobile’s impact on retail web traffic/sales during annual make or break inflection points like Black Friday through the holiday (40% of 2016 sales came from the mobile device; source, OuterBox, May 2017), it is all-important to deliver a seamless user experience not just across mobile, but all devices.

More than half of visits to mobile sites are abandoned after 3 seconds, so you don’t have a lot of time to provide your customers with the best user experience such that a purchasing event occurs across the mobile application. The structure of the mobile application brings extra challenges for the Performance Engineers – the way the information is loading on a mobile device, how it will react based on the network, bandwidth, etc. This can be summarized in the following three dimensions: mobile, network, and back-end.

 #1 The Device

Let’s start with the hardware itself. Just take a look at a mobile phone store to understand difficulties you will face. Every mobile device has its specifications (CPU, memory, screen size, resolution, etc.). This does not include the myriad Operating System options available.

How can you make sure that your application will deliver a smooth user experience when it runs under extreme conditions such as:

  • Facebook – Sending push notification during a business transaction
  • Receipt of SMS or call while the user is interacting with the application
  • Loss of signal (network)

#2 The Network

To have a clear view of how your application will react, you cannot avoid the network constraint reality. Your network presents one of the biggest challenges because the experience of 4G, for instance, could be completely different depending on the location, carrier, environment (I.e., crowded train station or airport). Most people think bandwidth is the main challenge. In reality, it’s latency, which Ilya Grigorik (Web performance engineer at Google), helps explain why.

The application will never enhance the network, so there is no way to avoid the constraint. That is why you have to consider the constraints the primary risk and measure the impact of your application under various network conditions.

In addition to the network having UX impact, so too does the back-end infrastructure. Slower users will stay longer within the architecture. The technical consequence of this – more concurrent sessions, greater memory or CPU consumed.

Native or hybrid applications are using API services from the back-end to collect data and allow rendering on the mobile device. It is crucial to validate UX with the help of a real device because limiting UX measurement through API calls would not be realistic. The rendering starts on the mobile device from the moment the device is getting API call response.

#3 The Back-end

It’s the back-end which hosts the entire architecture – API, database, etc. Testing the performance of the back-end is mandatory to properly size the capacity of the environment. Most of the projects are testing the performance of the back-end by running traditional load testing against API, or browser-based application. Most mobile applications utilize the common architecture of browser, third-party application. Most mobile applications utilize the common architecture of browser, third-party application. Therefore, limiting the load test of mobile users won’t be realistic because, in production, the back-end will serve mobile/browser users. It is mandatory to realistically define the load policy of each “user” generating noise (load) on the back-end.

Performance Mobile App Testing

To manage this, you will have to be prepared to answer some key questions:

  • How do I account for different devices and applications?
  • How do the different network characteristics impact application performance/user experience
  • How do I avoid back-end services problems?
  • How do I achieve the best user experience (measurable) possible?

Join us for an upcoming webinar co-hosted by Neotys and Bitbar on Wednesday, August 30th.

We’ve designed the webinar to help you answer these questions and to give you a sneak peek of the NeoLoad integration with Bitbar. We will cover the importance of mobile performance and functional testing, showcasing how the integration between NeoLoad and Bitbar will help you accelerate the delivery of high-quality applications to your end-users.

Webinar Details:

Wednesday, August 30th: 11 am CET (Paris, Berlin) or 1 pm EST (NY, Boston) now to have your seat!

About Bitbar:

Bitbar, a mobile DevOps company, specializes in automation of continuous mobile software delivery for enterprises. They help Global 2000 companies adopt DevOps in their mobile development to achieve fast, effective performance through automation rates of over 90%+ in every phase of their build-test-deploy-monitor process.

Bitbar solutions bring value to Agile and CI/CD methodologies with high-performance mobile software build, automated testing, instant deployment, and production monitoring process for native mobile application/game developers, and mobile websites that rely on revenue generation from their products.

Learn more at

Leave a Reply

Your email address will not be published. Required fields are marked *