Leading Media Company Accelerates Large-Scale Migration to AWS CloudFront with Confidence

Challenge

One of the largest Polish broadcasting companies was migrating to AWS CloudFront and needed to run high-volume, high-velocity performance tests in a multi-cloud environment with real-time feedback loops.

Solution

The company partnered with Soflab Technology, a leader in testing and quality assurance of ICT solutions and certified Neotys service partner, to use the NeoLoad Cloud Platform (in conjunction with AWS, Azure and Oktawave backup machines) as a fast and flexible way to run and re-run performance tests when tuning cloud configurations and in debugging/diagnosing system issues.

Solution

A major broadcasting company in Poland, one of the largest multimedia groups in Central Europe, was migrating its digital content delivery and services to the cloud, specifically AWS CloudFront. The company wanted to ensure that it delivered the best quality content to its subscribers all over the world — no matter what platform or device they were using — when facing peak load traffic.
The company’s IT environment was highly complex, and although it had mature Dev and QA processes in place, the QA team was looking for a modern performance testing tool that would allow them to replicate realistic, geographically distributed, very high production traffic just before expected prime time traffic peaks.
The company partnered with Soflab Technology, Poland’s leading provider of testing and QA services (and a Neotys certified service partner), which recommended Neoload to address the customer’s most important requirements:
  • High-speed, high-velocity preparation and modification of performance test scripts
  • Ability to easily use different cloud solutions to generate load
  • Easy to handle a very high volume of virtual users
  • Ability to view test results in real time during test iterations (including errors details in system responses)

Accelerating & Simplifying the Migration to AWS CloudFront

“The migration of even partial solutions to the cloud brings with it quite a large reconstruction of the existing system. In order for the mechanisms made available by cloud providers to work properly, it is not enough just to move application servers or databases to the cloud. In the majority of cases, migration to the cloud involves adopting some — or all — of the applications in such a way as to effectively gain all the benefits of the cloud solution,” says Michal Figiel, Head of Technical Testing at Soflab Technology.
With a large migration project like this one, there are a lot of complex risks to mitigate: user experience, cost, performance, etc. That’s why it’s important to efficiently compare performance baselines of what things were like before migration and what things were like after migration. Soflab and the media company needed to ensure that there was no regression in performance due to the migration and that applications were optimized so that traffic spikes don’t lead to increased cloud production costs as increased capacity “bandages” buggy apps.

“Dev changes were constantly introduced to the platform migrating to AWS CloudFront. Thanks to NeoLoad, we could react very efficiently to such changes.”
— Michal Figiel, Head of Technical Testing Dept., Soflab Technology
“Practically until the day the performance test started, dev changes were constantly introduced to the platform migrating to AWS CloudFront. Some of them modified client-server communication, which was recorded and parameterized during the preparation of test scripts,” says Figiel. “Thanks to NeoLoad, we could react very efficiently to such changes. Doing these tasks in NeoLoad is already significantly easier — and the workload much lighter — than with competing tools. Additionally, the built-in recorder allows us to record only the changed fragments of communication flow and parameterization mechanism. We could prepare a dedicated framework to significantly accelerate the work of subsequently modifying test scripts.”

Building a Large-Scale, Dynamic, Multi-Cloud Testing Environment

As Soflab’s Michal Figiel puts it, “Defining and building a cost-optimized load environment for cloud performance testing is not a trivial task. It cannot be assumed that a single load generator will support 500 or 1000 virtual users. That depends on many factors, such as the complexity of test scripts, amount of data downloaded/sent by the script, traffic characteristics and policy, predicted number of errors, and so forth.
“Before starting the performance test, we executed a so-called scaling test, which gave us a rough estimate of the hardware resources we’d need. So we already had initial insights and observations on what could have an impact on system performance, such as the size of the static files served. Eventually, after a series of scaling tests, we decided on a hybrid approach:

  • 1 Neotys Cloud controller (and additional as a backup)
  • 200 cloud machines located on the Neotys Cloud Platform
  • 200 backup machines from AWS, Azure and Oktawave, spread across various regions.

« Due to the ease of management, high reliability, dynamic infrastructure capabilities and predictable billing model, the Neotys Cloud Platform was chosen as the primary solution. However, because of certain project dynamics and in order to minimize risk, a backup environment based on three other cloud providers was additionally prepared. It’s also worth mentioning that we needed to divide available cloud load generators into zones so that the flood of responses from customers’ environments didn’t overload a single load generator.”
With over 70 locations around the globe, hundreds of servers at each location and no single point of failure, the Neotys Cloud Platform scales tests as large as the media company needed. NeoLoad is vendor-agnostic — all the major cloud providers are supported in the same platform — and optimizes testing infrastructure dynamically so that testing costs are predictable, kept under control and irrelevant to development costs. to run high-volume, high-velocity performance tests in a multi-cloud environment with real-time feedback loops.

Running Complex Performance Tests at Speed and Scale

Having run dozens of large-scale performance test cycles for the company not only for the migration to CloudFront but also just before any media event that may result in a significant increase in users, including traffic simulation of web, mobile and SmartTV platforms, Michal Figiel says that no “with the correct configuration and availability of the cloud, the services based on it are ‘almost impossible to kill.’”
“In migrating to the cloud there are many pitfalls that may be identified only by simulating real-life traffic. Large-scale test cycles uncovered, for example, common issues like VM provisioning thresholds and VMs getting up and running too late for an aggressive traffic increase, which usually happens in production during prime time or a breaking news broadcast,” explains Figiel.
“Although in theory cloud solutions allow for provisioning ‘unlimited’ bandwidth or computing power, in practice a number of requests, exclusions of certain protection mechanisms, or obtaining appropriate permissions are required in order to properly perform performance tests on a large scale. This is why we used the Neotys Cloud Platform for testing purposes — all reports, protection exclusions or permissions are covered by NeoLoad. From the point of view of executing tests, it comes down to just booking and running a test session. And the Neotys cloud controller and load generators are preconfigured to handle much more load than a single rented machine in any cloud solution. This makes it incredibly easy to build and maintain large load environments.
“All of the test iterations and configuration fixes were completed during just 5 nightly hours of testing.”
— Michal Figiel, Head of Technical Testing Dept., Soflab Technology
“We implemented the test cycle using a previously proven and repeatedly tested procedure. For the duration of the tests, a technical forum was set up comprising members of the media company’s tech team as well as from Soflab and Neotys. More than 20 test iterations were performed as part of the test cycle. Some of them ended with errors, others with slow system operation or its unavailability. After each iteration, we pinpointed the problems and implemented configuration or platform code change in order to achieve performance SLOs [service level objectives].
“In the final iteration, the achieved traffic significantly exceeded the assumed 200,000 concurrent users. Then we increased the generated traffic more aggressively. That was the final step before setting up logged users’ ramp-up policy from 0 to the maximum Neoload license value in 3 minutes. That was a final infrastructure stress test, when the transfer rate exceeded 90Gbps with throughput over 200,000 req/s, while maintaining full usability of the client portal and response times to the requests below 30 ms.
“In the end — apart from technical and organizational expertise, and NeoLoad being the best tool on the market — success came from the tight cooperation among people: the client’s IT team, Soflab Technology and an outstandingly supportive Neotys Team.
“It’s also worth pointing out that thanks to NeoLoad, precise preparations and tight coordination, all of the test iterations and configuration fixes were completed during just 5 nightly hours of testing.”
Company
A Leading Media Company
Industry
Television Broadcasting
Employees
1300
Location
Poland
DOWNLOAD THE PDF
❝

Dev changes were constantly introduced to the platform migrating to AWS CloudFront. Thanks to NeoLoad, we could react very efficiently to such changes.

❞

Start testing with NeoLoad, the fastest, the most realistic, and the most automated continuous performance testing platform.

Try NeoLoad
Request a demo