By now, you’ve probably heard about Spectre and Meltdown. In the last few weeks, there’s been many a blog on the patch topic:
- Spectre and Meltdown attacks against microprocessors
- More details about mitigations for the CPU speculative execution issue
- Microsoft says chip fix may significantly slow some servers
- Meltdown, Spectre bug patch slowdown gets real – and what you can do about it
As a Performance Engineer, this patch conversation does give me a little heartburn, but I know that I need to test before jumping to any conclusion. Therefore, I took one of our spare Intel Xeon W3550 servers and installed:
- OS: CentOs 7
- Demo application
What’s the Testing Scope?
To test the impact of the patch, I decided to run two different test types.
- CPU Behavior: Launching several types of “CPUburn” to measure patch conduct by generating CPU load, io, disk, and memory activity. This test doesn’t simulate the real activity of the server, but it is a simple way to measure patch impact.
- Traffic Simulation: Generating load test by reproducing web app traffic. The objective of this test is to detect if the patch is impacting the user experience of PHP/MySQL website.
Of course, those two tests will be executed before and after patch installation so that you can easily compare patch’s impact on the CPU or the end user.
Let’s review patch test results against Spectre and Meltdown
With the test complete, let’s compare the behavior of the CPU with the help of stress:
The behavior of the CPU System is almost the same after the patch.
Similar to the CPU System results noted above, the behavior of the CPU user is the same. From this first test, I am not able to make any conclusion on the impact of this patch on CPU behavior.
Load Test on Apache
By comparing the results before and after the test, NeoLoad is automatically providing a ratio on the expected KPI:
- Response time
The response time has increased almost 10% for key transactions.
Because of the increase in response time, I am not surprised to observe fewer transactions/sec for select key transactions. The application can generate -1% transaction/sec for each transaction.
In this scenario, the patch increased the CPU System usage by 16%.
On the other hand, the CPU user (Apache) is similar:
Because of the increase of the CPU System, the server has 15% less CPU availability post-patch deployment.
From the two tests conducted on my server, I was able to see two different behaviors: no impact; a small impact on the CPU and response time. So, it may be disappointing to you in my lack of providing specific test influence specific to the general behavior of this patch on the application. To be clear, the patch could have a negative impact on the behavior of your system depending on:
- CPU: Does the system have a recent or old CPU?
- OS: Depending on the operating system, the patch partially resolves the issue
- Application type
If you don’t want to jeopardize your production and users, I’d recommend running a load test to measure the impact of the patch on your application.