Load Testing Best Practices

What’s driving load testing in Agile and DevOps environments today? The ever-increasing share of digital technology is driving companies to accelerate the pace at which new services/features are released. At the same time, applications are offering richer functionalities with improved, more sophisticated user experience. The explosion of application reliability and speed is creating the need for greater complexity/intensity of the load testing practices behind them.

Research and development teams are faced with two challenges: the need to be able to test faster and more regularly. In response, they’re adopting Agile and DevOps methodologies. Some questions still remain – how do I integrate load testing, predominantly a manual discipline, into shorter cycles? Where can I incorporate automation to support accelerated delivery goals?

Performance anomalies can be costly to resolve. They can have a significant impact on an application’s code structure and/or architectural foundation. Early testing practices are a solution to minimize technical incident cost or to avoid a complete service failure for your users.

Shift Left and Component Testing – Driving Earlier Load Testing Execution

What’s the upside of faster, increased testing within shorter cycles? How about having the ability to quickly identify a performance problem upstream during development so that you avoid the typical practice of complex/costly post-sprint correction. Load testing earlier, continuously, and automatically is the right approach.

Service Level Agreement (SLA) Management

Conducting load tests earlier starts with proper documentation of user stories, which should always include all associated performance requirements (similar to functional testing approach). This involves identification and documentation of Service Level Agreements (SLAs) and load tests results analysis in relation to performance indicators such as:

DNS Resolution Speed
Response Time
Time-to-last-byte
System Uptime
The technical behavior of the infrastructure

A good rule of thumb when making sure performance is properly integrated into development work is to put SLAs in the user stories directly, as another dimension in your Definition of Done (DoD). In this case, a user story cannot be “completed” if the performance criteria are not met.

NeoLoad allows you to set custom SLAs. These thresholds are used to integrate load testing into the Continuous Integration process. It will condition pass/fail test status, determining whether to continue the automatic build process. Learn more about NeoLoad

Load Testing of Components

Validating code performance at the earliest stage means that testing starts well before the application is complete. Load tests should be run as soon as the most strategic components are available. The goal should be to load test APIs, web services and/or microservices representing the application’s life-giving organs.

NeoLoad is designed to make load testing as simple as possible. It is aimed at both specialist testers and developers and allows testing of APIs/components during the early stages of the development cycle. For more information, see our webinar Shifting Left: The Top 5 Reasons to Test APIs and Components Early

Service Virtualization for Realistic Component Load Testing

There is an interdependence with which an application’s components interact. Isolated load tests of each component preclude performance validation of an assembled system. As a result, the component testing approach offers two options:

Wait for component availability of those consumed by the load test
Implement service virtualization on components not yet available in order to make load tests more realistic, faster

NeoLoad interfaces with service virtualization tools such as Parasoft and SpectoLabs. Want to learn more? Check out our white paper, Using NeoLoad for Microservices, API, and Component Testing

Automation of Load Tests and Non-regression Performance Tests

When unit load tests of components are defined, developers can launch them automatically with the Continuous Integration server (E.g. during nightly builds). Build after build, these tests can check the component’s performance trend over time; this is a non-regression performance test.

Using the pass/fail status of the tests, the Continuous Integration server can automatically decide whether the integration process can continue, or if it must be stopped for immediate issue correction.

Since the status definition of a load test is closely linked to SLAs, it is necessary to define them precisely in order to be able to block or authorize the next steps in the build process.

With NeoLoad, you can automate your load tests within your Continuous Integration server, use SLAs to set the pass/fail status, and automate the continuation of the build process. Read our white paper about Agility and automation, At the Speed of Agile

How to Speed Up “Test-Analysis-Scenario Update” Cycles for Load Testing of Assembled Applications?

When the application’s components are assembled, integration testing is much more complex than unit load testing. In Agile environments, it is important to adopt the right strategy so that performance testing does not bottleneck the development cycle.

Building an Effective Strategy to Hedge Performance Risk

It is not possible to test 100% of an application. So, it is essential to define the riskiest areas of focus for load testing depth and intensity. In practice, 10-15% of the most successful test scenarios will identify 75-90% of the major problems.

For proper risk diagnosis, you must have a sufficient knowledge of the application’s architecture. Testers, application managers, and architects must work closely together to ensure this knowledge is shared.

Tools and Practices Responsible for Accelerating the Creation of Complex Load Test Scenarios

Gray box testing for the most important components of your application
Automating variabilization, correlation, and randomization
Functional test scenarios (E.g. Selenium) to speed up the creation of load tests

Neotys offers many tools to automate test creation, such as variabilization, correlation, and automatic randomization. With NeoLoad, you can import functional Selenium test scenarios for use during load testing. All you want to learn about NeoLoad’s features to design complex scenarios.

Ingredients of Effective Load Test Analysis

Agile developers need to understand the root of the performance issue for quick resolution. Diagnosis quality and analysis efficiency are essential load test contributions to the requirements of confidence.

Actionable data synthesis, E.g. whether they can withstand decision support
Understanding of the role/skills of the individual using the load test report
Use of APM/profiling tool to identify the piece of code responsible for performance regression

NeoLoad offers a test analysis capability allowing you to identify performance problem root, leaving more time to solve quickly. Read more about test results analysis.

Automating the Update of Your Load Test Scenarios

Updating load testing scripts often represents >50% of the teams’ effort. As the application evolves, it is often necessary to re-write the test scenarios completely. Here’s how to accelerate, or even automate, this maintenance activity nearly in-full:

Load test tool applied to detect code modifications; change part of the test
Rely on functional test maintenance to fully automate the update

Use NeoLoad’s automatic user path update feature to reduce the time it takes to maintain your scripts by up to 90%. For more about NeoLoad’s automatic user path update.

Integration of Load Testing in DevOps Toolchain for Continuous Testing

Collaborative Load Testing

Application performance is a concern for all members of the Agile team, not just those performing load testing services. It is important that your test results are accessible to everyone, at all times:

Each team member must be able to consult the test reports
It should be possible to monitor the performance measures in real-time during test execution (before the end of a load test, which can last several hours). Developers can start issue correction earlier.

With NeoLoad, all teams work together on the load test. Results are shared in a centralized data base similar to real-time monitoring results during the tests. Learn more about collaboration capabilities.

The Load Testing Platform’s Interaction with Your DevOps Toolchain

To enable fast and efficient acceleration of Agile and DevOps processes, it is necessary to integrate your load testing tool with the other solutions within your DevOps chain:

Incorporation into the Continuous Integration servers to automate load tests
Assimilation with functional testing tools to measure user experience/automate scenario maintenance
Alliance with APM tools to analyze tests at code level

NeoLoad offers ready-to-use integrations to facilitate successful toolchain interaction. You can create your custom integrations with NeoLoad’s API.

Essential Production Performance Monitoring Information Most Useful for Load Testing

Performance feedback gleaned during production with your APM tool are invaluable load testing approach refinement data. Armed with this insight, you can:

Evaluate load levels observed in production to adapt the testing strategy, reproduce realistic conditions
Measure actual service levels that condition SLAs in testing
Identify risk areas in your application

Load Testing to Performance Engineering

The art of load testing continues to evolve towards traditional performance engineering practices, yet application performance responsibility is still reserved for a small number of specialists today. This evolution, which will vary by business based on their individual performance testing philosophies/corresponding Agile/DevOps adoption, represents a profound shift in the methods, skills, and tools used by the tester. At the core, the skill set of the traditional performance tester requires expansion:

Knowledge of how to understand and evaluate the application’s architecture
Awareness of how to best incorporate automation into the toolchain
Communication and collaboration skill advancement to become trusted performance partner with others responsible for performance

In the end, the “new” performance engineer becomes the protector of performance throughout the project lifecycle. Their key responsibilities include:

Test strategy creation after execution of thorough risk analysis
Load testing automation enablement
Supporting developers with unit load test scenario creation
Monitoring of strategy implementation during production; establishment of incident reporting process

> Download NeoLoad and Start Testing Now
> Request a Personalized Demo of NeoLoad