I spent 30 days preparing for the NeotysPAC this year. Because the theme was “PAC” to the future, I wanted to do something that looked like it might have come from the future – as much as that was possible.
In 2016, I brought up the fact that HTTP/2 would be great for performance testing engineers to begin load testing because if it fulfilled expectations, web sites would be faster overall. While HTTP/2 did introduce some significant changes to the way browsers handle HTTP traffic, it still had leftover issues because the worse the network conditions get, the more it pushes the bottleneck one level lower into the TCP world. As soon as I started hearing about HTTP/3 (or HTTP/2 over QUIC) in early 2019, my ears began to perk up. I thought this might be the perfect time to try and get in on it at the early stages.
Technically, what I tested is better described as “HTTP/2 over UDP using QUIC draft 46.” The original version of QUIC first deployed by Google was HTTP/2 over UDP. However, this is not the same as the IEFT version of QUIC. After Google took QUIC to the IEFT, it was decided to turn it into a fully-featured transport protocol.
HTTP/3 using IETF QUIC was being standardized as I tried to do this experiment, and no full browser supported the client-side API libraries at the time. While the current HTTP/2 implementation in Chrome over Google QUIC “should” show similar performance characteristics to HTTP/3, this isn’t 100% certain. Let’s look at the differences between HTTP/2 and HTTP/3 from a protocol standpoint:
(Image Source: Daniel Stenberg)
What did I do? I set up my server on AWS, connected to a data source, set up a realistic WordPress site, added certificates and SSL (a must with HTTP/3), configured Cloudflare for DNS and CDN capabilities, and everything else required to have a legitimate HTTP/3 web site. At the time, the only web server supporting the latest version was LiteSpeed. Note: My original website (loadtesthttp3.com) has been removed since the research has been completed. The domain now directs people to the fully detailed research document I left behind.
Why did I do it? Because I don’t like to speak at conferences about theory and only things I have read. I committed several years ago that I would never speak on something that I didn’t get my hands dirty doing. I call this “experiential knowledge,” and for me, it is the only way I truly learn. Trying to test something so early in its life put me on the bleeding edge, and there is a reason they call it that. On the other hand, this is typically where a lot is learned.
This is a visualization of what I put together to make the testing possible:
This blog article can only be so long, so if you want to find out all of the details on how I set things up and all the testing details, there is a link to my research here: https://scottmoore.consulting/performance-testing-http-3-quic/.
I’ll spare you the gory details and summarize it: It does appear to me that the TCP bottleneck has been alleviated in comparison to HTTP/2. The waterfall images are a lot different. TLS negotiation will make a difference in overall performance by reducing the number of round trips for authentication by 50%. As expected, the worse the network conditions, the more benefit that will be seen. High-speed networks may not see a drastic reduction in overall page times. The image below represents 50 virtual users over a terrible satellite network connection:
The biggest challenge I see ahead is that this is a fundamental shift in moving from TCP to UPD transport and controlling packets on a layer above that. TCP is so rooted in web infrastructure that it could take a long time to get things moved over, and there are many bugs to find at that time since UDP isn’t used nearly as much as TCP. I am interested to see what that will look like.
By the time you read this, HTTP/3 will have matured even more, and some of the issues I ran into won’t be there anymore. Clients (i.e., browsers) and servers are starting to provide better support for it, so hopefully, when you get around to validating the performance, load testing tools will have native support as well, and your testing will be easy. I hope you will share your experience with others, and let’s find out if it does live up to its promise of a faster web.
Learn More about the Performance Advisory Council
Want to learn more about this event, see Scott’s presentation here.