Leading Media Company Accelerates Large-Scale Migration to AWS CloudFront with Confidence
One of the largest Polish broadcasting companies was migrating to AWS CloudFront and needed to run high-volume, high-velocity performance tests in a multi-cloud environment with real-time feedback loops.
The company partnered with Soflab Technology, a leader in testing and quality assurance of ICT solutions and certified Neotys service partner, to use the NeoLoad Cloud Platform (in conjunction with AWS, Azure and Oktawave backup machines) as a fast and flexible way to run and re-run performance tests when tuning cloud configurations and in debugging/diagnosing system issues.
- High-speed, high-velocity preparation and modification of performance test scripts
- Ability to easily use different cloud solutions to generate load
- Easy to handle a very high volume of virtual users
- Ability to view test results in real time during test iterations (including errors details in system responses)
Accelerating & Simplifying the Migration to AWS CloudFront
“The migration of even partial solutions to the cloud brings with it quite a large reconstruction of the existing system. In order for the mechanisms made available by cloud providers to work properly, it is not enough just to move application servers or databases to the cloud. In the majority of cases, migration to the cloud involves adopting some — or all — of the applications in such a way as to effectively gain all the benefits of the cloud solution,” says Michal Figiel, Head of Technical Testing at Soflab Technology.
With a large migration project like this one, there are a lot of complex risks to mitigate: user experience, cost, performance, etc. That’s why it’s important to efficiently compare performance baselines of what things were like before migration and what things were like after migration. Soflab and the media company needed to ensure that there was no regression in performance due to the migration and that applications were optimized so that traffic spikes don’t lead to increased cloud production costs as increased capacity “bandages” buggy apps.
Building a Large-Scale, Dynamic, Multi-Cloud Testing Environment
As Soflab’s Michal Figiel puts it, “Defining and building a cost-optimized load environment for cloud performance testing is not a trivial task. It cannot be assumed that a single load generator will support 500 or 1000 virtual users. That depends on many factors, such as the complexity of test scripts, amount of data downloaded/sent by the script, traffic characteristics and policy, predicted number of errors, and so forth.
“Before starting the performance test, we executed a so-called scaling test, which gave us a rough estimate of the hardware resources we’d need. So we already had initial insights and observations on what could have an impact on system performance, such as the size of the static files served. Eventually, after a series of scaling tests, we decided on a hybrid approach:
- 1 Neotys Cloud controller (and additional as a backup)
- 200 cloud machines located on the Neotys Cloud Platform
- 200 backup machines from AWS, Azure and Oktawave, spread across various regions.
Running Complex Performance Tests at Speed and Scale
“In migrating to the cloud there are many pitfalls that may be identified only by simulating real-life traffic. Large-scale test cycles uncovered, for example, common issues like VM provisioning thresholds and VMs getting up and running too late for an aggressive traffic increase, which usually happens in production during prime time or a breaking news broadcast,” explains Figiel.
“Although in theory cloud solutions allow for provisioning ‘unlimited’ bandwidth or computing power, in practice a number of requests, exclusions of certain protection mechanisms, or obtaining appropriate permissions are required in order to properly perform performance tests on a large scale. This is why we used the Neotys Cloud Platform for testing purposes — all reports, protection exclusions or permissions are covered by NeoLoad. From the point of view of executing tests, it comes down to just booking and running a test session. And the Neotys cloud controller and load generators are preconfigured to handle much more load than a single rented machine in any cloud solution. This makes it incredibly easy to build and maintain large load environments.
“In the final iteration, the achieved traffic significantly exceeded the assumed 200,000 concurrent users. Then we increased the generated traffic more aggressively. That was the final step before setting up logged users’ ramp-up policy from 0 to the maximum Neoload license value in 3 minutes. That was a final infrastructure stress test, when the transfer rate exceeded 90Gbps with throughput over 200,000 req/s, while maintaining full usability of the client portal and response times to the requests below 30 ms.
“In the end — apart from technical and organizational expertise, and NeoLoad being the best tool on the market — success came from the tight cooperation among people: the client’s IT team, Soflab Technology and an outstandingly supportive Neotys Team.
“It’s also worth pointing out that thanks to NeoLoad, precise preparations and tight coordination, all of the test iterations and configuration fixes were completed during just 5 nightly hours of testing.”
Dev changes were constantly introduced to the platform migrating to AWS CloudFront. Thanks to NeoLoad, we could react very efficiently to such changes.