Holidays are high-stress times for everyone. Retail operations come under their maximum pressure during this period, causing resultant stress for both IT operations employees and the front-line, customer-facing order-taking applications that they manage.
In these kinds of applications, any page response delay that customer experiences in the app can mean the difference between growth and loss for the retailer.
A lousy shopping experience can also lead to a firestorm of negative conversation on social media, which can quickly tarnish the brand at a critical time of year.
Holiday retail “winners” have realized that they must rally around a customer focus, creating seamless shop-to-purchase experiences regardless of the device the customer is using to shop. A significant part of that seamless experience is the response time of the application, how fast it will respond to the customer.
Holiday periods and the congregation of customers that are shopping at the same time can adversely affect this response time of an application, even if it is normative at other times of the year.
Testing the Load
One of the arrows in the performance engineer’s quiver is the load test. Load testing means a performance-related testing process that places a simulated demand on software, web or mobile applications to measure responses and systems’ behavior under both normal and anticipated peak load conditions. The testing process is where any bottlenecks that appear in the test results are analyzed and (hopefully) causes for them are ascertained.
But normative load testing may not do the job for the holiday season situation. The parameters used for the test may be too forgiving, too easy.
Users realize that the performance testing conducted needs to be realistic so that they can prepare for peak holiday volume. However, typical testing techniques may not be cost-effective. The more the virtual user count needs grew, the more the user may have to pay. Eventually, it becomes cost-prohibitive to test under real peak scenarios.
Not only that, but while trying to replicate peak loads, the testers may get a lot of false positives. So, even if the testing team takes on the excessive cost, the effort may be for naught.
A Way Out
What is needed for this situation is stress testing. As the name implies, it’s a testing process that is designed to push an application’s environment to its breaking point. It’s done this way so the performance engineers can gain an understanding of the upper limits of capacity within the system.
A stress test, therefore, becomes especially important for e-commerce apps around big holiday seasons or online tax filing services during tax season, for example. Any situation that will cause a situation-focused app to become more busy than usual will benefit from stress testing.
Stress testing will benefit the organization by revealing the application issues that only become apparent under extreme conditions. Further, it allows testers to determine the software’s robustness and so ensure that the system will fail and recover acceptably.
A stress test should first identify test objectives, critical scenarios, the workload you want to apply, and the metrics to be tracked. After the test cases have been created, a stress testing tool will be needed to simulate the required load for each test case and then capture the performance metrics which will result.
Stress tests can also show other kinds of interrelated problems.
Examples of these would include:
- Synchronization and timing bugs
- Interlock problems
- Priority problems
- Resource loss bugs
- Memory leaks
- Data loss & corruption
Each of these is all the kinds of things that show up when development is used within the production system in the real world. Each segment may function fine independently on its own, but when placed into use with the rest of the system dependencies can show up.
An excellent performance testing tool will realistically simulate thousands of users, analyze the application’s performance as well as monitor the servers under load. Neoload exemplifies the wants and needs from such a tool. It gives actionable insights from high-level dashboards and detailed metrics rather than just data that needs to be further analyzed to be useful.
Not only that, but it can be combined with cloud storage through the NeoLoad AWS service so that data storage can be expanded to meet testing needs without requiring excess capacity (and the capital cost of it) “just in case.”
Flexible and cost-efficient stress testing is within reach of any DevOps or Agile team through this kind of method. The security of knowing that a project has been protected from failure in peak periods by a concerted testing regimen can only ensure a favorable outcome when the real world meets the planned one.
Larry Loeb has written for many of the last century’s dominant “dead tree” computer magazines including BYTE Magazine (Consulting Editor) and the launch of WebWeek (Senior Editor). Additional works to his credit include a seven-year engagement with IBM DeveloperWorks and a book on the Secure Electronic Transaction (SET) Internet Protocol. His latest entry, “Hack Proofing XML,” takes its name based on what he felt was the commercially acceptable thing to do.
Larry’s been online since UUCP “bang” (where the world seemed to exist relative to DEC VAX) and has served as editor of the Macintosh Exchange on BIX and the VARBusiness Exchange. He lives in Lexington, Kentucky and can be found on LinkedIn here, or on Twitter at @larryloeb.