Open Enrollment Launch: 5 Lessons Learned, 5 Years Later

Open Enrollment is the period from early November until mid-December of the calendar year when US citizens and other qualified non-citizens can sign up for medical insurance under the Affordable Care Act (ACA). During this time millions of people will log onto to select/update medical coverage for the new year.

Today,’s operations are effective and efficient. But, this wasn’t always the case. When the ACA was first introduced in 2013, Open Enrollment was nothing short of a disaster. The site couldn’t handle the volume of users trying to sign up. Millions were left staring at a spinning web page when the site was struggling to process requests.

Unfortunately, it took a significant failure to learn the lessons required to get the portal to a place where it could support the request volume effectively. Takeaways had to do with technology and the managerial process. Turns out that managing the expectations and implementation of a technology is equally as important as getting it to work.

The following five lessons learned describe the most significant outcomes from initial failure to the eventual success of  teaching that can be useful to any company who’s creating mission-critical software, particularly under less the optimal circumstances.

1. A Big Bang Will Blow Up Big

After the Patient Protection and Affordable Care Act became law in 2010, the Centers for Medicare and Medicaid Services (CMS), the agencies responsible for building, had three years to get the website up and running. It was an arduous task that went well beyond name and address association with a particular medical plan. Plans and fees varied by state. Also, the system needed to interact with a variety of other private and government systems. The laundry list of features that are necessary to be implemented kept growing as did the number of contractors employed on the project. However, few if any at CMS understood the task’s enormity or the details associated with it.

If that wasn’t enough, the site had to be rolled out, perfectly on the first day of operation. The luxury of an incremental release plan option was not possible. Those in charge wanted a big bang release, and boy did they get one. Suffice it to say the system blew up for all of America to see. It was a technical and PR nightmare.

Had the scope of work been scaled back, as many were recommending, and an incremental approach to development and testing been taken, the site’s debut would not have been perfect, but at least an acceptable starting point. Sadly, this was not the case.

The lesson(s) learned? To start, as history has shown, the odds of releasing a big system perfectly from the getgo are less than favorable. Meanwhile, the magnitude of failure potential is likely to be directly proportional to the size of the system. When a big bang release blows up, it blows up big. Releasing a system incrementally, with a limited number of features dedicated to each release cycle betters an all at once approach. Had gone the subsequent release plan route they’d have achieved more successful results.

2. System Interoperability Counts was not designed to be a standalone system. It couldn’t be. The Medicare and Medicaid products were government-provided, the remaining policies from the private sector or state programs. As noted, the website was being asked to be more than a name and address order taker proving/verifying identity and eligibility. It also had to present appropriate insurance policies allowing the user to choose, then facilitate payment once a plan was selected. In the background, many of these tasks involved interacting with systems outside of the environment.

System interoperability adds a significant layer of complexity to infrastructure, more so when systems operate at the scale of governments and insurance companies where physical interaction might require working with legacy systems still running “old” mainframes.

It’s easy to second guess today, but it seems that no one properly anticipated the amount of work, money, and innovation necessary to facilitate the level of system interoperability needed to make functional. Failure seemed inevitable.

What was learned from this? The minute a system has to go outside of itself to implement features, the risk of failure increases dramatically. This is not limited to interacting with an old-fashioned legacy system; modern systems run this risk too. Look at what happened to data curation company, DataSift when Twitter refused to renew its data access license. The company had to jump through hoops to survive. In the grand scheme, DataSift provides a useful service but not one that can result in harm or death if the site goes offline. For, access to external data is a matter of life and death, thus proving the value of system interoperability.

3. Make the Big Complex Task Smaller & Monitor Closely

Essentially,’s failure was that it was not able to deliver on the fundamental task allowing users to find, get, and pay for affordable health insurance successfully. Moreover, management neglected to recognize and understand the impact that the smaller tasks had on full system operational success. The following excerpt from the Inspector General’s Report sums it up perfectly.

“Because CMS did not always provide adequate contract management and oversight for Federal marketplace contracts, (1) contractor delays and performance issues were not always identified, (2) a contractor incurred unauthorized costs that increased the cost of the contract, (3) contracting officers in all Government agencies did not have access to contractor past-performance evaluations when making contract awards, and (4) critical deliverables and management decisions were not properly documented.”

In other words, was a project being run without adequate project management. Things had just run amok.

A grand vision is necessary, but if you don’t have the wisdom to embrace that the devil is in the details nor possess the expertise to implement the systems necessary to manage them, failure is certain. Consider what expert Len Schlesinger, Professor, Harvard Business School had to say in an interview about the initiative in 2016, “The vast majority of the people who were involved in the process didn’t understand the task.”

Those who are successful in bringing large projects to fruition have a talent for breaking the primary complex task (which define the project overall) into multiple large but manageable tasks that can be monitored and executed according to plan. Again, an easy to say now sentiment, but indeed something that could have saved

4. There’s No Getting Around Good, Fast, and Cheap

Passing a law is hard. However, paying for it is infinitely harder, particularly in the face of political opposition. The ACA statute provided money for states to build exchanges, but there was no funding provided by the bill for the Federal portal. As a result, instead of being a well-funded undertaking, had to be creative about its financing procurement to bring the federal features online.

There is an old project management saying that suggests the notion of having things good, fast, and cheap, that you can never get all (picking your ideal two is recommended). Every project manager dreams that he or she can deliver an extraordinary product before the deadline, below budget. Since only two of these wishes come true, you’re likely to be faced with any of the following challenges that force the prioritization to chose two:

  • Good and fast don’t come cheap
  • Fast and cheap don’t produce good
  • Good and cheap take time

Those at the helm of the non-negotiable implementation of sought the trifecta only to miss on all. History has shown that you cannot get around, Good, Fast, and Cheap and must be okay with 66% achievement. It’s a lesson that every certified PMP knows, and who has likely learned firsthand.

5. Testing Only Counts Once You Can Test

Because was developed in a quasi-waterfall manner, with minimal iterations, the opportunities to thoroughly test the site were limited. Without access to a fully functional website, critical tests such as standard load testing could not be performed with predictable regularity. Is it a surprise then that one of the first things to go wrong with the site was system performance degradation under load? It makes sense; you can test only those parts of the system you can access. If the full system is not available, you can’t test it.

The lesson? When designing and implementing a system, make sure that you enable such that you can perform standard system level tests regularly throughout system evolution. This includes, but not limited to, load, performance, regression, and penetration testing. Early iterations of the system will most likely test poorly. But, having full, testable access to a system to a poorly performing system is still more ideal than having no access until public release.

Putting it All Together

Anyone associated with’s first Open Enrollment period will tell you that the experience is not something they choose to relive. Its primary source of disaster stemmed from poor management. “They were running the biggest start-up in the world, and they didn’t have anyone who had run a start-up or even run a business,” says David Cutler, Harvard professor and health adviser to Obama’s 2008 campaign.

Fortunately, things have improved. In 2012, Todd Park became CTO of the United States where he introduced Bain & Company alumnus, Jeff Zients as the primary problem solver. The government removed the financial, procurement, and contracting constraints that had hindered the project from the beginning. Zients brought in technologists from Google, Amazon, and Facebook – some paid, some volunteers  to get on its feet.

The system has improved and shown steady usage volumes over the last three years. Despite the window shortening from 90 to 45 days in 2017, the system was built to accommodate nearly the same amount of volume over half the amount of time a feat that cannot be understated.

The initial experience of Open Enrollment on was anything but optimal mistakes were made and lessons learned. Today, is successfully fulfilling its basic mission: to make insurance available to every American citizen regardless of any pre-existing condition. If only those involved had exposure to a lesson learned point-of-view before go-live, perhaps these results would have come sooner and with less cost.

Learn More

Discover more load testing and performance testing content on the Neotys Resources pages, or download the latest version of NeoLoad and start testing today.


Bob Reselman 
Bob Reselman is a nationally-known software developer, system architect, test engineer, technical writer/journalist, and industry analyst. He has held positions as Principal Consultant with the transnational consulting firm, Capgemini and Platform Architect (Consumer) for the computer manufacturer, Gateway. Also, he was CTO for the international trade finance exchange, ITFex.
Bob’s authored four computer programming books and has penned dozens of test engineering/software development industry articles. He lives in Los Angeles and can be found on LinkedIn here, or on Twitter at @reselbob. Bob is always interested in talking about testing and software performance and happily responds to emails (

Leave a Reply

Your email address will not be published. Required fields are marked *