Last Minute QA Checklist for Black Friday

Black Friday and Cyber Monday are just around the corner, which means that by now there are two types of people who are fully immersed in the holiday shopping season: department store music selectors and web application testers.

Yep, the most active shopping days of the year will be here before you know it. So your website better be ready for the deluge of online shoppers who’ll be banging on your digital door.

We all know what can happen if your site isn’t ready. We hear about another epic fail every year. In 2013, it was Moto X – remember that one? Motorola announced that it would offer cheap phones for a short time, and so many people wanted in on that deal that the website crashed and a ton of orders were left unfulfilled. The company’s CEO personally apologized for the incident, and the company held a “do-over” on the deal.

And then there was the Kohl’s disaster in 2012, when the website crashed for several hours on Thanksgiving, after the department store promised its customers early Black Friday deals. Website visitors, ready to spend cash, couldn’t get past the front page message, “We’re sorry for the delay!”

Obviously, you don’t want this to happen to you. So we put together this simple checklist to help you prepare.

Lock Down Your Environment

Errors arise from changes. By now you should have your production environment locked down with a very clear and controlled process for managing changes. Keep a backup of your site available – hopefully it is all contained with source code control anyway – and monitor your site for unauthorized modifications. Any changes that do need to be made should be tested thoroughly, reviewed by peers, and sent through a committee before being released into the production environment. Then they should be tested again.

Begin Regular Load Testing

With your environment locked down, you should have high confidence that your site functions as expected – that is, the code is working. So now you want to focus on scale by performing series of load tests in both your QA environment and your production environment. In getting ready for Black Friday you’ll want to try lots of different scenarios, focusing on suspected bottlenecks in the system. It’s a good idea to leverage a cloud testing platform to provide some geographical realism to your scenario as well, that way you are accurately simulating the many locations that will be accessing your site.

Communicate With Your Monitoring Team

Too often, QA and Operations organizations don’t speak as much as they should. Black Friday is a great time to break down those walls. Look for ways to bolster communication. Something as simple as a daily report can really work wonders. Collaborate with your counterparts on the monitoring team to identify key metrics across both QA and Production and disseminate them through a daily preparedness email. These open channels will help you put in place more complicated forums for working together, such as a war room to address issues in real-time, should they happen.

Set Up Synthetic User Monitoring

If you haven’t put in place synthetic user monitoring, you should strongly consider it. A synthetic user is like a load test in production. You simulate a user, script it to execute a variety of tasks on the product website, and monitor the actual performance of those tasks. You get real data about how the production environment is behaving, but without having to rely on actual users who may experience problems. Work this into your joint QA-Operations plan for a proactive way to avoid problems.

Know What Promotions Are Being Planned

Talk to your marketing team about the promotions they are going to run. For example, if they are going to run a deal like the one Moto X ran, it would help if you knew about it in advance. That way you could bulk up on your infrastructure, get your operational teams in place, and do everything you can to prepare for all the likely scenarios. Keep a specific calendar of all these promotions, and test each one in advance of Black Friday. You’ll want to know exactly what you are dealing with.

Create a Social Media Monitor

Users like to complain. And if they are trying to get a deal and can’t, they’ll lash out on Twitter or Facebook. Set up automated monitors and alerts to scour the comment stream and look for customers who are having problems. You’d be surprised how big an impact an attentive customer service rep can make to a user’s experience. But even more importantly, these signals could be an early warning sign indicating a problem is happening somewhere.

Create a Contingency Plan

So what happens if things go really bad? You can’t afford to be shut down over a weekend when consumers spend somewhere around $2B – so you have to have a backup plan. And you have to have a backup plan for your backup plan. Make sure you have a failover site set up and make sure it’s been thoroughly tested. Have trained staff at the ready to jump into issues and handle any problems that arise. Make sure your contingency plans are well-documented, circulated, and clear.

And Finally, Test Again.

Remember, you can never be too prepared. When everything else is in place, keep testing. Go through a few more scenarios. Push the limits a little farther. Approach the system from a slightly different angle. The more you push your website, the more confident you’ll be.

When it comes to Black Friday and Cyber Monday, the best offense is a strong defense. Put the work in up front to test your site and be prepared for anything that might happen. And good luck!

No Responses
  1. November 19, 2014

Leave a Reply

Your email address will not be published. Required fields are marked *