The ControlPath Compliance Suite is a purpose-built solution for compliance and enterprise risk management. The ControlPath solution automates the compliance process, and allows organizations to leverage their compliance work efforts across multiple regulations (Sarbanes-Oxley, Gramm-Leach-Bliley, HIPAA, PCI, and FISMA), security standards such as ISO17799, and internal policies and standards.
“Our software application is a 3-tier java-based application with a tomcat-based web front end. Our UI is very dependant on [a href=”http://www.neotys.com/product/technologies-ajax-load-testing.html”]ajax[/a] traffic and is an asynchronous conversation with the webserver. Also, because our application can be split vertically across up to 3 servers (Web, app, DB), we have many factors that can affect end-user performance.
We were looking for an all-purpose load-testing tool to be able to do scalability testing, peak performance testing, code performance tuning, and to be able to publish specs on application performance in certain configuration scenarios. We needed:
- Allowing sets of users to be defined and run in parallel with different authentication parameters.
- Randomization of post and querystring data.
- Capturing page response time, http post/get data, and the stats of the server
- Providing simple reports in addition to full details of errors and activity
- The tool had to scale to large numbers of virtual users AND run across several load-generating machines.
- Visual set of tools to manage scripts, virtual users, test settings
- Graphical output of test results (graphs, charts)
- Real-time visualization of load tests in progress
- Simplicity, Simplicity, and yet more Simplicity (the tool will be used by all manor of non-engineering staff at times)
Our team did not have time to dedicate resources to performance testing, so a tool that allowed engineers to “put on the performance hat” and run a test without the need for a dedicated, trained resource was important.”
“We chose NeoLoad almost immediately because it was a breeze to setup. Then, as we were working with it, we found a goldmine of functionality that let us start using NeoLoad to accomplish debugging and troubleshooting efforts far beyond simple load tests. Special-case race conditions, randomization of usage patterns, all of these were available out of the box and were a huge help to the testing. Furthermore, the NeoLoad tool was very very easy to configure for even the more complex tasks. Where other tools required manual script writing, the NeoLoad tool captured and played back simple to exceedingly complex scripts, and allowed us to tweak the scripts flow with a host of logic operators, all with drag & drop ease.
When we started heavy-duty testing, we found that the analytics captured by NeoLoad was awesome – we could capture all manor of statistics in harmony with the other data captured by the test, to watch for places where many different factors aligned. NeoLoad went even farther by automatically saving all our test runs, so that we never had to keep track of complex piles of “test output data” elsewhere on our disks.
Another hug win for us was the ease at which the “basic” reports can be published. We have two audiences for test data output: Engineering and Customers. Engineering requires detailed analysis available with just a few clicks through the NeoLoad GUI, while our customers only require the basic “end results” reports – which are seamlessly exportable. Other solutions we looked at required much more “report template” work to get results formatted for customers.
Our product is used by large national and international banks, so assuring that all the t’s are crossed and i’s are dotted in the engineering side is critical. NeoLoad plays a very important role not only to us but to our customers, helping us deliver a better product and providing the kind of performance analytics that our customers require in their purchase and operation decision making.
And to add icing to the Cake, the NeoLoad pricing was dramatically cheaper than other products in its Enterprise class. NeoLoad’s entry costs were low enough to allow us to make a department-level decision to purchase it, which saved untold weeks or months of higher-level decision making. With any other enterprise-class load-testing tool, we would have had to go as high as CEO approval for such an expense, which would have laid waste to our timelines. With NeoLoad, we could choose, purchase, setup, and get the results we needed within the time that purchase approval would have taken for its competing products.
Our testing period was the shortest in history – we were able to setup and run a load test in under an hour even though we had just downloaded the product for the first time. In fact, we were able to make our buying decision the very same day as our download because we accomplished so much so quickly.”
Test Results and Added Value
Test Results and Added Value
“Our load testing allowed us to publish specs that helped ease some customer issues with performance limits, as well as later performance tests used as race condition tests allowed us to isolate and error that occurred so rarely that no human testing had ever caught it.
Beyond that, we’ve used load tests to generate massive amounts of data to do scalability testing of our data stores and techniques. These tests help isolate SQL command overuse, index tuning, and a host of other data-storage related issues that would normally require much more man-power to accomplish.
We were also able to publish metrics that none of our competition had yet done: a load test combined with artificially injected TCP delays on our servers to see how TCP packet delays between Web, App, and DB servers would affect end-user performance – the results were actually quite amazing and we’re the only company in our space that publishes stats on the impact of packet latency between servers of the application. This kind of in-depth testing has helped us get ahead of our competition in the eyes of our clients as a more thoroughly tested and more stable application.”