Comparing Load Testing TCO of LoadRunner and Neotys NeoLoad

Introduction

There are many cost elements that should be assessed when calculating the total cost of ownership (TCO) for a load testing solution. Since every organization’s load testing requirements are different, you need to take into account the specific context of your performance and load testing process and environment to calculate a true cost profile and a realistic TCO that is relevant to your organization.

NeoLoad Load Testing Overview

NeoLoad is not only a less-expensive alternative to  LoadRunner; NeoLoad is a more-efficient and faster load and performance testing tool for web, mobile, and other digital applications. This is especially true for applications developed in fast-paced environments practicing DevOps or Agile methodologies.

How NeoLoad is Different from LoadRunner

Faster Test DesignAutomate_3

Most testing tools (including LoadRunner) have a GUI with record and playback capabilities that require some coding to further customize scripts, but NeoLoad is different because there’s no coding required at all. Correlations can be run automatically at the end of each recording. Parameterization and randomization of variables take just a few clicks, and logical actions are all drag-and-drop. Former LoadRunner customers tell us that using NeoLoad results in up to 10 times faster test design compared to using LoadRunner.

Script Maintenance Automation

Load Test Conti_2That’s why NeoLoad has a user path update feature that allows you to update a designed script with a quick recording and automatically keep variable extractors, think times, SLAs, logical actions, JavaScript, validations and more from the original user path design. This makes the script maintenance process up to 20 times faster so that performance testing can keep pace with current Continuous Integration/Continuous Deployment environments.

Better Support for Web Technologies

Neotys has set a consistent track record for enhancing NeoLoad with support for new web Built-in-Techno_NLprotocols months (and sometimes years) ahead of LoadRunner. This improves TCO because you don’t have to spend time and resources hand-coding a workaround to include these new technologies as they become available – NeoLoad supports them out-of-the-box. Of course, the real test of a great load testing tool is not just that it’s able to test a particular technology – it’s how well your tool can test that technology.

Many current NeoLoad customers were once LoadRunner customers who made the switch because NeoLoad was able to more accurately capture and replay traffic from applications utilizing technologies like HTTP/2, WebSocket, GWT, Oracle Forms, Flex, adaptive bitrate streaming and more.

Out-of-the-box Support for Mobile Applications

NeoLoad comes out-of-the-box able to perform proxy recording of native mobile applications, mobile device simulation, and network virtualization; there is no expensive mobile package to buy. More importantly, because mobile is fully integrated into NeoLoad, you don’t lose time setting up mobile simulations, as you must do with LoadRunner. You can even run tests with virtual mobile users on your web applications with just a few clicks.

Lower Up-Front and Ongoing Costs

The cost of a NeoLoad license is typically around one quarter the cost of a LoadRunner license, even taking into consideration the typically aggressive discounting of licensing for LoadRunner.  When you combine this with the ongoing savings and huge efficiency gains you get with NeoLoad, the return on investment is significantly higher for NeoLoad vs. LoadRunner.

Our Findings

NeoLoad’s 3-Year TCO is 3x less expensive (or about 1/3 the cost of LoadRunner). Our calculations show that the savings from the first 18 months of NeoLoad ownership more than offset the entire 3-yr total cost of ownership for NeoLoad.  It would take 18 Months to pay back a full three years’ TCO investment in switching to NeoLoad with operational cost savings

Conclusion

Creating a test environment that mirrors your production environment 100% is practically impossible. Therefore, by definition, we are NL_vs_LR_ROI_Blog_Image1retrieving unrealistic performance results from our tests due to the key differences between the testing and production environments.

Most companies avoid load testing in production due to its potential impact on actual users’ activities and their data. You can reduce the impact of production system testing on actual application users by following best practices like testing during off-hours on off-days, testing before a release, testing during a maintenance window, performing read-only database operations or implementing service virtualization and carefully monitoring the test execution.

Load testing exclusively in production is better than nothing, but limiting the testing effort to the production environment is not recommended because it won’t allow you to properly tune or size the application before testing in a high-risk setting. Instead, testing in production should often be viewed as the last validation to guarantee the user experience by including CDN in the testing approach. More than load testing, applying a proactive application monitoring is recommended to detect performance issues before the real production users experience them.

Learn More

For more information, download the Neotys White Paper, “Comparing TCO of LoadRunner and Neotys NeoLoad.”

Leave a Reply

Your email address will not be published. Required fields are marked *