This document discusses Netflix's transition to a cloud native architecture and lessons learned that may be applicable to other companies. The key points are: 1. Organic migrations don't work and big changes need to be planned from the start. 2. Embracing managed cloud services and standardizing what to centralize makes migrations easier. 3. Security, controls, and costs need consideration upfront to avoid issues later. 4. Having guardrails for decision making is important but not being too constrained. 5. Learning from failures and getting comfortable with change is part of becoming cloud native.

1. What Worked for Netflix
may not Work for You
Subbu Allamaraju
@sallamar
https://subbu.org
July 18-19, 2018

2. 75+
countries
93
currencies
67
languages
© 2018 Expedia Group 3

3. © 2018 Expedia Group
600M+
m o n t h ly v i s i t s i n 7 5 +
c o u n tri e s
1.6M+
a c t i ve c o r p o r a t e
t r ave l e r s
150K+
o f f l i n e trave l age n ts
55M
a n nu a l t r ave l c a l l s
10K
a f f i l i a t e s
500+ a i r l i n e s
500K+
h o t e l p ro p e r t i e s
25K+
u n i q u e a c t i v i t i e s
1.5M
v ac ati o n re n tal s
150+
c ar re n tal c o m p an i e s
Lots of people
going lots of paces
stats provided as of 12/31l/2017
4

4. 9 billion pricing and
inventory updates daily
>750 millions
searches per day
15 billions + flights
searches annually
© 2018 Expedia Group 5

5. The Cloud
Native Recipe
© 2018 Expedia Group 6

6. © 2018 Expedia Group 7

7. Contemporary cloud native trail map and landscape
(developed and promoted by https://cncf.io)
© 2018 Expedia Group 8

8. Our Recipe
© 2018 Expedia Group 9

9. © 2018 Expedia Group 10

10. © 2018 Expedia Group 11
!
• Which ones are easy to migrate an
and which are hard to migrate?
• Are there systems that you’re afraid
to touch?
• Anything to retire?
• Are there monoliths?

11. Stateless monoliths Stateful monoliths
Large shared databases
No clear view of dependents
Too critical to change quick
Low rate of change
Historically centralized
Hairball code - dev/build time hell
Every one rides the same release train
Some parts want to move faster
Low ability to change fast
Shared ownership
© 2018 Expedia Group 12

12. Replatform
Untangle for high rate of change
High pain – high gain – high motivation
New services – cloud ready
Lower barrier of entry – high velocity – high
learn rate
Lift and shift
Needed rate of change vs time to replatform
Low pain – low gain – low motivation
Move and tune
Most as quickly as you can, then tune iteratively
Pace the change to reduce risk
Deprecate
Code and practices that you don’t want to carry
forward
No one size fits all
© 2018 Expedia Group 13

13. © 2018 Expedia Group
P
E E
N
D
EN
C
IE S
D
14
Develop an understanding of
sequencing the migration

14. © 2018 Expedia Group 15
What does the target state
mean to you?

15. Cloud native is …
1. Embracing cloud service ecosystem
2. Internalizing cloud economics
3. Operating securely amidst higher rate of change and
choice
4. Becoming resilient by building redundant fault domains
and getting comfortable with intentional failures
© 2018 Expedia Group 17

16. Building blocks for evolving architectures
EC2
API
Gateway
Athena CloudSearch
EFS
EMR
Kinesis
QuickSight
Route 53
SES
SNS
SQSCloudWatch CloudTrail
Config
Data
Pipeline
DMS
Direct
Connect
Directory
Service
KMS
Lambda
DynamoDBEBS ElastiCacheRDS RedshiftS3
CloudFront
EC2 Container
Service
EKS
ElasticSearch
Service
GuardDuty
DynamoDB
Accelerator
VPC Shield Glue
SageMaker
Neptune
MQ
© 2018 Expedia Group 18

17. Cloud native is …
1. Embracing cloud service ecosystem
2. Internalizing cloud economics
3. Operating securely amidst higher rate of change and
choice
4. Becoming resilient by building redundant fault domains
and practicing intentional failures
© 2018 Expedia Group 19

18. Cloud spend measurable
A team tracking their daily and
7-day moving average spend
Unallocated costs
Services and
usage types
Cost of test
environments
Cost of legacy
instances
© 2018 Expedia Group 20

19. Cloud spend is also manageable
1. Tuning the architecture for better price-performance
2. Shedding unnecessary/unwanted traffic
3. Scaling down passive regions
4. Adjusting SLAs for your analytics jobs
5. Using cheaper tier options for S3 and EBS
6. Adjusting data retention policies
7. Using cheaper resources for test workloads
8. …
© 2018 Expedia Group 21

20. See Cloud Optimization Circus for more.
© 2018 Expedia Group 22

21. Cloud native is …
1. Embracing cloud service ecosystem
2. Internalizing cloud economics
3. Operating securely amidst higher rate of change and
choice
4. Becoming resilient by building redundant fault domains
and practicing intentional failures
© 2018 Expedia Group 23

22. Write down your controls
(SOX, PCI, IT general controls, …)
Automate controls
(Accounts,VPCs, connectivity rules, access controls,…)
Automation baked into continuous
delivery pipelines
Audit your controls
© 2018 Expedia Group 24

23. Cloud native is …
1. Embracing cloud service ecosystem
2. Internalizing cloud economics
3. Operating securely amidst higher rate of change and
choice
4. Becoming resilient by building redundant fault domains
and practicing intentional failures
© 2018 Expedia Group 25

24. Low velocity
Resilient
High velocity
Fragile
High velocity
Resilient
Low velocity
Fragile
ResilientFragile
Rate of change
© 2018 Expedia Group 26
?

25. Part 1: Redundant fault domains enable traffic shifting and
minimize in-place firefighting
Faults inside a fault domain
don’t cascade to other fault
domains
Any request that enters a
fault domain is fully served
inside the fault domain
Vegas Rule
See Fault Domains and theVegas Rule
© 2018 Expedia Group 27

26. Part 2: Intentional failures reveal hidden failure modes
o Automation is never
complete
o Configuration drift never
goes away
o Silent failures are not
uncommon
© 2018 Expedia Group
o Start small
o Build redundancy and practice
failovers before trying larger
failures
o TestVegas rule through region
exit/failovers
Practice and get comfortable through the hybrid phase
Motivators Practice
28

27. Cloud native is …
1. Embracing cloud service ecosystem
2. Internalizing cloud economics
3. Operating securely amidst higher rate of change and
choice
4. Becoming resilient by building redundant fault domains
and practicing intentional failures
© 2018 Expedia Group 29

28. © 2018 Expedia Group 30

29. Our Guardrails
1. Automate almost everything
2. Practice finance governance
3. Comply to controls
4. Implement fault domains and theVegas rule
© 2018 Expedia Group 31

30. © 2018 Expedia Group 32

31. © 2018 Expedia Group 33
Knowing where to go is easy.
Leading there is the hard part.

32. © 2018 Expedia Group 34
Everything is gray
No reasonable option …. Or too many options
Don’t know where to start
Don’t know who should solve
Current org/team structure not setup to solve
These are symptoms of ambiguity.
Ambiguity gets you stuck.

33. © 2018 Expedia Group
1. Recognize when you’re stuck
2. Challenge constraints
3. Learn to say “yes”
4. “Learn it all” over “know it all”
Getting Unstuck
35
Growth mindset can help

34. Lessons
Learned So far
© 2018 Expedia Group 36

35. 1. Organic migrations don’t work. Focus on big boulders from
the start, and plan.
2. Embrace managed or multi-tenant services on the cloud.
3. Learn quickly what to centralize and standardize and what
not to.
4. Accommodate security and controls from the get go.
5. Costs will hit you in the face. Build cost governance into the
culture.
6. Articulate guardrails to guide decision making – but don’t let
them constrain choice.
7. Learn to disambiguate quickly.
© 2018 Expedia Group 37

36. Q&A
@sallamar
https://subbu.org
© 2018 Expedia Group 38