Next Wave provides business strategies and software tools for Direct Selling companies worldwide by equipping their sales force with online business and communication tools that drive selling, recruitment and retention.
We were looking for a tool that could easily record and maintain scripts with easy variable extraction and customization, and that could seed the script with custom queries from target database and record performance statistics from within same tool as executing the test.
Scripts were difficult to develop and maintain, seed data was extracted using a manual and protracted process, stress test result data was collected by a number of different tools and analysis and report generation was tedious and error prone.
Prior to NeoLoad we used: PureLoad, JMeter, OpenSTA.
A web application providing tools to a sales force, providing different types of tools to Consumers, Salesmen and Administrators. The application modules are interconnected and are exposed with different UI and functionality depending on user type. Some of the modules are CMS package, Product Catalogue, Order Entry, Contact Management, Sales Event management, Email Campaign processing, etc.
Protocols/technologies tested: Http/https, ajax, webservices.
NeoLoad provides a very powerful set of features that are controlled by an easy to use and intuitive interface. The scripts are easily recorded and modified to keep pace with application development. The monitors allow consolidation of data from multiple, otherwise not related systems. Access to database from within the script execution allows the campaign to resemble production scenarios very closely.
QA team members create and maintain scripts to closely match their Use Cases. The scripts are used initially for automated QA Acceptance testing, before performance testing is initiated.
During a typical campaign we would first executed a scenario slowly stepping up the number of users, while monitoring all systems involved, app servers database, network, etc. If there are clearly identifiable issues with the database or network observed, the recorded data is handed over to appropriate team to be handled. If there are issues with application servers, they are instrumented with profiling tools, and any recorded issues with application are handed over to the development teams.
After a number of iterations, when a maximum load is established at a comfortable level, a burn-in campaign is executed, where the maximum load is maintained for a number of hours.
Test Results and Added Value
Because of the simplicity of the process, we were able to separate the recording and maintenance of the scripts to the QA team, and only involve the technical personal during the load testing campaigns.
The monitors allow to map and correlate the performance of separate systems into one graph. Before such statistics were gathered separately, and correlation was more art than science.
As result of the testing, we are easily able to identify the parts of the application where tuning is necessary, whether its code or SQL related. We were additionally able to tune the hardware setup for out SQL server, specifically to the type of traffic expected during production peaks.
For example; improved software quality, increased testing productivity, cost savings, improved customer satisfaction/service, reduced risk, shorter time to market, competitive advantage, other?
Load Testing in an integral part of our development cycle. It ensures that the software would be able to handle the usage spikes, thus greatly reducing the risk for our clients and guaranteeing a consistent customer experience. The cost savings are achieved by reduced hours support teams have to spend tracking down performance issues while they are occurring in production.
We will continue to utilize NeoLoad for both automated QA and load testing as we do today.