This post is part one of a four-part series collectively covering the following topics:
- Part 1 – Introduction to Performance Testing
- Part 2 – Establishing a Performance Testing Strategy
- Part 3 – Modeling Performance Tests
- Part 4 – Performance Test Execution
Introduction to Performance Testing Overview
Applications are being released to the market with increased complexity every day and done so using shorter development cycles. This requires an embrace of Agile development and testing methodologies. Performance, as part of the global user experience, is now the primary determinant of application quality.
“Old school” sequential projects marked by static Qualification/Implementation/Test phases, which push performance testing until the end of the lifecycle, are sure to experience a performance risk along the way. This proposition is no longer acceptable by today’s application quality standards. This blog series will provide practical information on how to execute efficient performance testing in this new and more demanding environment.
The State of Complexity in Modern Apps
One of the main drivers behind this shift to modern load testing is the growing complexity of the IT landscape:
- Most users are using mobile, thin clients, tablets, and other devices to access information.
- Complex architecture(s), simultaneously shared by several applications, is being built.
- New technologies offer a range of solutions (AJAX framework, Rich Internet Application (RIA), WebSocket, and more) aimed at improving the applications’ user experience.
Historically, applications have been tested to validate quality across several areas: functional, performance, security, etc. Testing phases answer to user requirements and business risks. However, the dialogue has changed; the conversation is no longer about quality, it’s about user experience. The user experience is a combination of look-and-feel, stability, security, and performance.
Performance: Imperative to a Successful User Experience
Performance is the factor in the success of the user experience. This is due to advances in technology, architectural complexity, and the locations/networks of the users. Load testing was a nice addition to the development process but has quickly become an essential testing step. Load and performance testing answers the following questions:
- Is the application capable of handling a certain number of simultaneous users?
- Is the average response time for pages acceptable under this set load?
- Does the application revert to normal behavior after a load peak?
- How many users can the application handle while maintaining acceptable response times?
- What is the load threshold above which the server(s) begin(s) to generate errors and refuse connections?
- Do(es) the server(s) remain functional under high load or does it crash?
Like any testing activity, performance testing requires proper methods and logic.
When an application passes performance testing but then fails in production, it is often due to unrealistic testing. In these cases, it’s easy but misguided blame to point at the testing itself or the tools used to execute it. The typical problem resides with test design, especially when done without the correct basis. It’s necessary to ask, “what did we need to know which, if we had the information ahead of time, would have allowed us to predict this failure before production?” In other words, “how can we deliver and execute an efficient performance testing process?”
Phases of a Load Testing Project
Methodologies like Agile and DevOps allow for application creation to quickly address customers’ needs. The practices of updating project organization require closer collaboration between teams. Within these methodologies, the project lifecycle is broken out into several sprints, each responsible for delivering a part of the application. In this environment, the performance testing process should follow a specific workflow (as noted at the top of the post).
A performance testing strategy should be implemented at an early stage of the project lifecycle. The first step: Performance Qualification. This defines the testing effort of the whole project. An “Old School” approach to performance testing would force the project to wait for an assembled application before performance validation would begin. In a modern project life cycle, the only way to include performance validation in an early stage is to test individual components after each build and implement end-to-end performance testing once the application is assembled.
In part two of this four-part series focused on putting performance testing to work for you, we will explore “how best to establish your performance testing strategy.”