Abercrombie & Fitch Co. Partners with NeoLoad for Annual Holiday Readiness Performance Testing
Abercrombie & Fitch Co.’s search for a new performance testing partner began in 2009 on the heels of its annual holiday readiness evaluation, which had identified gaps in its legacy solution. The wish list was simple – the company wanted a partner that was both cost-effective and flexible/scalable enough to facilitate realistic performance/load testing out-of-box. Also, the A&F team wanted script maintenance simplicity, a functionality the brands did not have before working with NeoLoad.
Abercrombie & Fitch Co. Context
Headquartered in New Albany OH, Abercrombie & Fitch Co. is a leading specialty fashion retailer with over 900 stores worldwide, and three brands under management, including Hollister Co. and abercrombie Kids.
In the retail sector, survival is based on the ability to adapt quickly and seamlessly to customer demands, whether location, culture or trend-driven in nature. A&F has always been at the forefront of reading and reacting to the market to meet customers’ evolving needs, whether adjusting its target customers’ profile preferences, keeping store operations informed, enhancing the shopping experience, or delivering innovative marketing/branding to adapt to evolving customer engagement patterns.
Abercrombie’s digital footprint includes 40+ eCommerce desktop and mobile sites, a volume that necessitates constant focus and effort to maintain performance at every turn. Responsible for portfolio performance, including maintenance of the 80 associated scripts, is Ajay Pedhagandham, Quality Assurance Manager, an industry veteran of nearly 14 years. His purview consists of performance testing team management (ownership and functional) and performance test automation. Most importantly, he’s been at the helm leading A&F’s holiday readiness testing team for the past four years.
Why is realistic performance testing so crucial to Abercrombie & Fitch Co.?
- Any page response delay, even for a second, can mean the difference between growth and loss. To prepare for peak, the team has to simulate tens of thousands of orders per hour to ensure readiness.
- The brand’s customers are socially active influencers – a bad shopping experience can lead to a firestorm of negative conversation, which can quickly tarnish the brand.
Performance Testing Challenges with Legacy Tool
Given the value A&F has placed on performance, including allocation of a dedicated team, it’s no surprise that some of the historical challenges the company encountered before NeoLoad included cost and coverage support – technology and customer service. Ajay recalls some of the early pains before the switch in 2010:
- “As we were growing and expanding our application delivery practices, which included increased global reach, we knew our performance testing coverage had to support this growth. Our performance testing had to be conducted realistically so that we could prepare for the peak holiday volume. The more our virtual user count needs grew, the more we had to pay. It was becoming cost prohibitive to test under real peak scenarios.”
- “Time and again, when we were trying to replicate peak load, we would get a lot of false positives. The reality is, true holiday readiness was not easily achievable before NeoLoad.”
Abercrombie & Fitch Co.’s Holiday Readiness Successes with NeoLoad
Once the fall 2010 transition to NeoLoad was complete, A&F was able to focus on site performance. Coincidentally, this integration happened during the annual holiday readiness effort. Despite the late-year transition, Abercrombie was ready for holiday 2010 – a testament to the focus and commitment of the QA team’s ability to manage multiple projects while ensuring flawless execution.
It is important to note that peak order volume testing with NeoLoad has met the “ success” threshold each year since 2010. Scripting time/test maintenance has reduced by over 80% (or, seven days down to 1).
- “We’re now walking into the war room having achieved our performance testing goals already, and this enables us to focus on back-end systems.”
- “In 2014, this extra time allowed us to identify and fix an issue we would never have caught, nor had the time to address, had we been focused on holiday load testing goals.”
Pedhagandham cites the other key growth drivers that the NeoLoad product has been able to help the A&F team achieve:
- Increased confidence when they say “ready!” – More frequent realistic testing = regular tuning on the fly.
- “ We no longer just test during the holiday, it’s happening throughout the year.”
- Increased visibility across the organization – More people are paying attention; performance appreciation goes beyond the Performance Testing team.
- A strong partnership with Neotys support staff – “ Neotys takes feedback seriously, and is committed to processing and support improvement when requested.”
What’s Next for Abercrombie & Fitch Co.?
At press time, Ajay and team are preparing to enter yet another holiday readiness war room. They will be doing so with excitement and confidence that they are ready to go as a result of the support from NeoLoad. The team’s also looking forward to more learnings and increased site-wide optimization next year.
A&F sees the near future holds a deeper integration of NeoLoad within its Continuous Integration toolchain. It has set its sights on being able to regularly embrace Agile/DevOps methodologies vs. the old – world “ only when cost and time would allow ” frequency.
… peak order volume testing with NeoLoad has met the “success” threshold each year since 2010. Scripting time/test maintenance has reduced by over 80% (or,7 days down to 1)