KPIs to Watch When You’re Load Testing in a Hurry

Have you ever found yourself in a testing time-crunch?

You know the situation… the development team was late delivering code, but your CEO adamantly opposes any change in the release date. All you’re told is: whatever it takes, get that application tested as quickly as possible.

Here’s a very familiar scenario one Redditor posted, explaining the pressure he gets put under when timelines are tight:

“My [developers] always want to run their tests against development hardware, and have me come up with some magic multiplier to get results from production. No such number exists. It’s non-existent because you had two completely different configured environments that are not going to have the same bottlenecks. If you reach 100 users in dev, all you know is you can probably handle 100 users in production. Past that, no guarantees.”

So what are the best practices when it comes to load testing in a hurry? What should you focus on in a time crunch? Here are three simple KPIs you should monitor and pay attention to when you’re in a rush.

1. Concurrency

Concurrency testing is one of the key metrics related to load testing. Concurrency testing is performed to identify the defects in an application caused by multiple users accessing the application simultaneously, and it measures performance attributes related to those use cases.

This KPI addresses how many users can be on the system at once. If too many users are trying to access the same functions at the same time, you may see an impact to response time, throughput, or even deadlocks that prevent users from interacting with the application further.

However, solely focusing on concurrency as the only KPI worth paying attention to can be misleading – and unfortunately this is too common when load testing in a hurry. Concurrency is just one factor – and you’ll have a much better picture of how your application performs under stress when you combine it with the transaction rate, as you’ll see the next point.

2. Transaction Rate

Pages/sec or CPU usage are solid operational metrics to monitor, but when it comes to business goals a more important factor to know is whether your application can process the 10,000 product searches per hour needed to support the business’s economic objectives.

This is where transaction rate steps in. Choose as your KPI a business transaction to monitor, for example, the number of searches, purchases, or other business-generating activities that user complete in a given period of time. If you monitor concurrency and transaction rate together you are essentially monitoring overall load placed on your site. As stated above, concurrency rate looks at how many users your site can handle while transaction rate looks at how many actions users complete to keep the business running.

These two KPIs then feed into the third and last KPI you should always monitor…

3. User Experience

Your users’ experience is of the utmost importance. The Internet is a big place and your users are constantly comparing their interactions with your site against interact with sites of all types (not just your direct competitors). It’s your job to monitor how well your site performs for them. Are you meeting their expectations? Or are you letting them down?

When looking at user experience, there are two aspects to monitor: availability and performance. The performance aspect looks at how fast the site loads, how fast operations are completed, and how fluid the interaction feels. The availability aspect concentrates on users’ access to specific functions, which can range from an unresponsive button click to a 404 “Page Not Found” error.

As a general rule of thumb, the top 5 things users will do on your site will account for most of the load generated on your servers. Pay attention to how your users behave with respect to these 5 tasks when you are load testing in a jam, and you’ll have covered most of the bases related to delivering a great user experience.

There Are No Shortcuts When it Comes to Monitoring Performance

While there aren’t any real shortcuts to performance testing, there are some things you can do when you are under the gun to finish a performance test. These 3 KPIs will definitely help guide you in the right direction.

No Responses
  1. February 5, 2015
  2. July 22, 2015
  3. October 15, 2015
  4. January 26, 2016

Leave a Reply

Your email address will not be published. Required fields are marked *