HTTP/2 initially seemed promising for reducing latency, but caused intermittent slowdowns for the client's critical assets. Testing revealed HTTP/2 increased median page latency by 20-30% compared to HTTP/1.1. While HTTP/2 performs well in some scenarios, it was not suitable for delivering critical assets globally due to variance issues. Over time the performance of HTTP/2 has improved, but introducing artificial latency still results in a 2.3x slowdown, so HTTP/1.1 remains a better solution for this client's critical path resources.

1. HTTP/2: Discovery and Woe
Bryan McLemore
Principal Infrastructure Architect, Client Systems & Infrastructure
or how HTTP/2 nearly derailed our latency initiatives…

2. Let’s Set the Stage

3. Background
• Speed is a feature
• We’re talking about the end user experience
• Measured by collecting data from browser APIs
• We look at the 90th percentile
• Our latency measurement has, and will continue, to evolve
Latency

4. Background
Circa Early 2017….
• Web client was delivered from a single AWS Region
• Several important regions had very high p90 latencies
• Our organization got a mission: reduce latency globally
• After experimentation, we kicked off a greenfield replacement of the web client
• And that meant examining and rebuilding everything

5. Background
• Released in 2015 as RFC 7450
• Multiplexes requests over a single TCP
connection
HTTP/2
• Originally released in 1997 as RFC 2068
• Single request per TCP connection
• Limited to six connections per domain
HTTP/2 HTTP/1.1
“[HTTP/2 gives] Speed, speed, and more speed, providing for a better user experience.”
~ Patrick Stox, Nov 27, 2015
“Adopting HTTP/2 speeds up your website without any changes to your existing codebase.”
~ CDN Documentation, Sep 10, 2018

6. Story Time!

7. Intermittent Slowdowns
• We received a report of intermittent
slowdowns
• Initial investigation showed no issues
• But the reports kept coming
• Critical CSS and JS files were randomly slower
than they should have been
The Report

8. Intermittent Slowdowns
• Reproduction completely eluded us
• What made this machine unique?
• He was receiving an unexpected CDN IP
• VPN DNS gave us edge IPs from all over the world
• Eureka! Finally able to reproduce
Reproduction

9. Intermittent Slowdowns
• Non-trivial latency
• A browser loading an html page with multiple critical assets
• Browser cache has to be empty
• Reproduction impact is inconsistent
Reproduction Criteria

10. Intermittent Slowdowns
• WHHHHHHY?
• Happened against multiple CDNs and even no CDN
• Browser selection didn’t really matter, but Chrome
had the worst impact
• Machine performance contributed to the level of
impact
• Tested without ssl, slowdowns vanished. Progress!
Wrath of Khan Moment

11. Intermittent Slowdowns
• HTTP/1.1 to the rescue!
• Slightly slower in an ideal situation
• Significantly more consistent
• Provided you have six or less requests
Solution
Wat?

12. Intermittent Slowdowns
• All testing so far has been anecdotal
• Needed a large scale latency experiment
• We got one in September 2017
Validation

13. SOURCE | Twitch Internal Data - Sept 2017
20% to 30%
HTTP/2 increased page latency
Intermittent Slowdowns
Proof is in the Numbers

14. Intermittent Slowdowns
• 90th percentile heavily penalizes variance
• HTTP/2 was very bad for our page latency
• Some workloads are amazing with HTTP/2
• Conventional wisdom misled us
• How to meet our latency goals
What did we learn?

15. Intermittent Slowdowns
• About a year has passed…
• Does seem better than it was, but still has a long way to go
• Testing with ~ 200ms of latency creates a 2.3x slowdown
Fast forward to today

16. Is http/2 best for your
critical assets?

17. Bryan McLemore
@kaelten
@kaelten
mclemore@twitch.tv