NeoLoad Office Depot has improved testing productivity through a reduction in scripting time and maintenance of test scripts by approximately 66%
Office Depot Europe
Office Depot is the number one reseller of workplace products and services in Europe with around 6,000 associates and two main brands: Office Depot and Viking. The company uses a mix of company-owned operations, joint ventures, and strategic partnerships, with shops in France and Sweden.
Office Depot began operating in Europe in 1990, starting first in the United Kingdom. In March 1999, our Viking brand launched its first European website for consumers and businesses in the UK, a pioneering moment in the industry. Office Depot has subsidiaries in Austria, Belgium, Czech Republic, France, Germany, Ireland, Italy, the Netherlands, Slovakia, Spain, Sweden, Switzerland and the UK
Office Depot Europe is currently in the process of migrating all EU Legacy websites onto a single Hybris web platform. For the initial pilot roll out in 2016 it was decided that JMeter would be used for load testing, due to the relatively low volumes required for testing.
After the successful pilot Go-Live, the QA team was tasked with carrying out a technical Proof of Concept to identify the preferred load test tool for all future performance testing. This tool should meet the technical criteria defined by the QA team, and should address some of the drawbacks we found when using JMeter, such as:
- Limited reporting functionality built into the tool
- Stability problems under load and extended durations of time
- Complex interface which did not allow for cross training amongst the team
- Limited ability to integrate Selenium to measure front-end performance
- No possibility for client-side mobile testing within JMeter
How was web application load testing accomplished before NeoLoad?
Before NeoLoad was purchased by Office Depot Europe, the QA team predominantly used JMeter for Load Testing. For this testing, JMeter was hosted in the AWS cloud.
Please describe the application you tested
The application is a Hybris web application with a three-tier architecture. The Production and Performance Test environments are hosted externally to Office Depot Europe within the Hybris cloud.
Which protocols/technologies were tested?
The protocols used for load testing are HTTP/HTTPs and JMS, in future this will also include WebSphere MQ.
Why did you choose NeoLoad? What alternatives were considered?
The key reasons for choosing NeoLoad were:
- NeoLoad satisfied the majority of the technical criteria identified before the Proof of Concept (e.g., ease of use for script development and scenario development, built-in analysis and reporting, available hosting platforms)
- Ease of use for cross training amongst the QA team
- Customisable reporting to share with key stakeholders
- Integration with other toolsets used within the team, such as Selenium and Jenkins
The main alternatives that were evaluated as part of the PoC were LoadRunner / StormRunner, JMeter, and SOASTA.
Can you briefly describe your load testing campaign?
Our teams work in an agile environment, with some performance tests executed against each sprint. The QA team also use Selenium for automation testing, and Jenkins for CI, which we are in the process of integrating with NeoLoad for front-end performance statistics and more frequent performance testing against builds during each sprint.
How did NeoLoad contribute to improving your load testing?
NeoLoad has contributed to improving our load testing in the following ways;
- Quicker execution of tests within a sprint
- Improved reporting to stakeholders
We are integrating Selenium scripts within our NeoLoad tests to quantify front-end performance alongside the back end performance statistics we get from NeoLoad.
Additional members of the team are now cross-trained and running tests in NeoLoad to allow earlier scripting within a sprint.
What is NeoLoad’s value to your business?
The value of NeoLoad to our business is:
- Improved application stability. For example, tuning of JVM garbage collection settings, reducing the requirement for additional infrastructure in production
- Improved testing productivity through a reduction in scripting time and maintenance of test scripts by approximately 66%
- Earlier identification of performance issues, resulting in improved customer satisfaction feedback
- Ability to cross train and upskill other team members
How will you use NeoLoad in the future?
Our plan for NeoLoad is to:
- Continue regular testing in sprint
- Integration of Jenkins and NeoLoad for automated testing within the sprint
- Integration of Selenium and NeoLoad to measure front-end performance
- Cross training amongst all QA team members in performance testing
Any additional comments?
During our Proof of Concept and early adoption of NeoLoad, Neotys provided invaluable support and guidance when required and were proactive in working through issues when encountered.
Our teams work in an agile environment, with a number of performance tests executed against each sprint […] NeoLoad has contributed to improving our load testing in quicker execution of tests within sprint