Netflix carefully plans holiday capacity to ensure availability while maintaining efficiency. A cross-functional team works with engineering teams to understand capacity needs through bottoms-up and tops-down modeling. They prepare for spikes by coordinating large service migrations to optimize "trough borrowing" and manage failover capacity. The goal is to fulfill their charter of ensuring efficient, available cloud capacity so engineering can prioritize innovation and availability.

1. Netflix Confidential
Santa Cloud: How Netflix Does
Holiday Capacity Planning
August 9th, 2016

2. Netflix Confidential

3. Netflix Confidential

4. Netflix Confidential
● Additional complexities
○ Volatile customer traffic
○ Option to do regional failover (Chaos Kong)
○ No budgeting process
● Can’t run too lean or too wasteful
Capacity Tightrope Walking
● Holiday season = “let’s not get fired” season

5. Netflix Confidential
● Charter: “Ensure availability of cloud capacity in
an efficient manner, allowing engineering
organizations to further prioritize innovation
and availability.”
● Cross-functional team amongst:
○ Engineering
○ Data Science
○ Finance
Cloud Capacity Planning

6. Netflix Confidential
● Match methodology for the environment
○ Bottoms-up
○ Tops-down
○ Highly iterative
● Engagement with the largest service teams
● Evaluate migrations and coordinate changes
Holiday Preparation

7. Netflix Confidential
● One of our largest services changing hardware type
● Service profile:
○ ~20% of our total footprint
○ Autoscales ~65% in the course of a day
○ Runs in all regions
○ CPU bound, memory intensive
Case Study: API Service

8. Netflix Confidential
Changing The Trough
Reservation line

9. Netflix Confidential
Managing Failover Capacity
● Evolution of the Chaos Kong
● Cascading failovers
● Who gets capacity?

10. Netflix Confidential
● Through detailed planning:
○ Maximize trough borrowing
○ Highest chance of capacity availability
● Charter: “Ensure availability of cloud capacity in
an efficient manner, allowing engineering
organizations to further prioritize innovation
and availability.”
Our Role In The API Migration
● “Rinse-repeat” for the other large service teams

11. Netflix Confidential

12. Netflix Confidential
Questions