4 Emerging Trends in AI & Load Testing

In the world of load testing, Artificial Intelligence (AI) is a game changer. In the old days, load testing was about getting a group of engineers into a room and having them set up test environments, run some scripts, then pore over the results. It was a daunting process. Fortunately, automation came along and did away with a significant portion of manual test administration components. Environments provisioned automatically; scripts ran according to schedule or on-command.

Still, there was a good deal of manual human interaction when it came time to evaluate results. It wasn’t unusual for test evaluators to export results into an Excel spreadsheet for manual “good” vs. “bad” separation. The ability to anticipate future load problems was still a long way off.


But that was then, and this is now.

AI is populating the technological landscape at increasing rates. Technologies such as Apple’s Siri, Microsoft’s Cortana, Samsung’s Clara or Google’s Assistant are now commonplace. In the IT world, IBM Watson is the big player. AI is here, and it’s bringing a new dimension to performance testing in general and load testing in particular. As the cost of AI drops, its power increases – a win-win situation for those in QA. The best part, it’s only going to continue to improve. As you’ll see below, AI is going illuminate load testing practices in the very near future. Let’s take a look some of the reasons behind this.

1. AI Will Analyze Test Data Better and Faster

There’s more to test data than just the gathering of results. Even something as simple as load testing a heartbeat URL using a lean HTTP request is going to produce a significant amount of side data. Between data log access and the data emitted from within the web server (E.g., system-detected information when there’s a redirection behind the heartbeat URL), there are plenty of server requests at-play. And of course, you can’t forget the results data. That’s a lot of data that can be used to provide profound insights into a load testing scenario, or at least that’s the goal. The reality can sometimes be different, especially when. There’s just too much data available.

AI will assuage this shortcoming. AI is designed to process more data, faster. Modern AI runs on hardware optimized for intelligent data processing. The machine learning algorithms AI uses are developed to get smarter as data consumption increases.

The ramifications regarding performance and load test are significant. Soon AI will be able to automatically provision environments (plugging into the system resources where monitoring is required), conduct the testing collect the data for analysis. Humans will set parameters at the highest of levels, while AI will implement the details provided, ultimately removing the human intervention bottleneck.

2. AI Will be Able to Identify Performance Trends with More Granularity

When it comes to identifying performance trends, the devil will be in the details, particularly when it comes to system anomaly discovery. AI works well with specifics and will help increase a company’s ability to cull through the volumes of test data to diagnose anomalies and trends. AI will make it so that tests can be conducted with finer granularity and thus, produce more detailed test results. A finer grain of results allows AI pattern recognition features to identify, and in many cases, anticipate performance bottlenecks. As AI learns more, it will be able to sense when an application infrastructure will reach its limits adjusting the environment accordingly.

3. AI Will be Used to Create Smart Service Level Agreements

Service Level Agreements (SLA) have kept more system admins up at night than a room full of crying babies. A usual scenario is that software as service (SaaS) provider, internal or third party, makes a promise to stakeholders to maintain 99.99% uptime. The stakeholders accept the commitment only to find actual system uptime is only 98.99%. This one percent difference increases downtime risk from 8.5 hours to more than two days! Yes, load testing was conducted before the SLA published, but only reflected examination at a single point in time. With later testing execution and improvement resulting in degradation discovery, the SLA never revised. The SLA promised one set of expectations; the real experience was entirely different – if only the SLAs were SMART.

SMART is an acronym for Simple, Measurable, Attainable, Realistic, and Time-bound. Presently, most SLAs don’t measure up to this. There are too many moving parts to monitor. It’s the fundamental shortcoming of human powered systems. However, once you include artificial intelligence into the mix, things have a way of changing. AI keeps track of all the moving parts. It can be wired into a system’s monitors to analyze performance to a fine grain. AI can detect operational trends in a system directly. When system performance changes, hopefully always for the better, the SLA can adjust in real time. A smart SLA allows service providers to set stakeholder expectations according to current as opposed to past system behavior. Also, Smart SLAs enable test designers to create load tests according to the latest performance data available.

Smart SLAs are a different way of doing business, and AI will make them possible.

4. AI Will Help Predict Danger Before it Happens

Predicting impending malfunctions in IT systems is hard. Trying to induce them in a system test is even harder. As previously mentioned, there’s just too much going on for humans to consider. Being able to anticipate system failures is AI’s wheelhouse.

AI can build a usage model for an application under test which can be iteratively applied to predict future behavior. For example, AI can use the SMART techniques to perform trend analysis on hard disk drives. This analysis is then applied to predict drive failure, applying failure predictions to aid system redundancy planning. AI goes beyond just gathering data and doing a calculation on that data. It will be able to connect the logical dots to alter full system behavior.

AI also can assist in determining the prominent usage patterns in systems. Once identified, AI can incorporate these patterns into the design of Virtual User scenarios used in performance tests.

Not only will AI increase the reliability of predicting system breakdown in general operations, but test engineers will also be able to use AI to create test strategies focused on recognizing dangers most prone to appear.

Putting it All Together

Artificial Intelligence brings test automation to a new level. Its predictive and inferencing power allows test automation to move past simple task executing defined in a script. Now, AI is a thinking partner in the test automation process. As it continues to take over more of the everyday, tasks of test design and implementation, test engineers will be able to focus more on the creative execution of software testing.

The idea of collaboration between artificial intelligence and human creativity was science fiction just a few short years ago. Today, it’s the road ahead for developing and delivering software that matters.

Learn More

Discover more load testing and performance testing content on the Neotys Resources pages, or download the latest version of NeoLoad and start testing today.


Bob Reselman 
Bob Reselman is a nationally-known software developer, system architect, test engineer, technical writer/journalist and industry analyst. He has held positions as Principal Consultant with the transnational consulting firm, Capgemini and Platform Architect (Consumer) for the computer manufacturer, Gateway. Also, he was CTO for the international trade finance exchange, ITFex.
Bob’s authored four computer programming books and has penned dozens of test engineering/software development industry articles. He lives in Los Angeles and can be found on LinkedIn here, or on Twitter at @reselbob. Bob is always interested in talking about testing and software performance and happily responds to emails (tbob@xndev.com).
One Response
  1. May 29, 2018

Leave a Reply

Your email address will not be published. Required fields are marked *