1.
Containers at Netflix - An
Evolving Story
Sangeeta Narayanan
Engineering Manager @Netflix
@sangeetan

2.
InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/netflix-containers

3.
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Presented at QCon San Francisco
www.qconsf.com

4.
Going Global
Recent Launches: Italy, Spain, Portugal
Early 2016: Korea, Hong Kong, Taiwan, Singapore

5.
Growing BIGGER

6.
Winning Moments of Truth

7.
Getting BETTER

8.
Jan
Apr
Jul
O
ct
2015

9.
Jan
Apr
Jul
O
ct
2015

10.
Jan
Apr
Jul
O
ct
2015
A/B Tests
New features

11.
Jan
Apr
Jul
O
ct
2015
A/B Tests
New features
Device Launches

12.
Going FASTER

13.
Cloud Native Architecture

14.
Automation & Tooling

15.
Culture

16.
New Considerations
Diversity - Users, Tech stacks, Workloads
Dev Ex - Dev iterations, Debugging, Packaging
Rapid Delivery
Resource Sharing

17.
Dev-centric packaging &
reproducibility
Infrastructure abstraction
Faster dev iterations
Resource sharing
Why Docker?

18.
Docker @Netflix - Use Cases
Algorithms Engineering
Data Pipeline
Edge Engineering

19.
Docker @Netflix - Use Cases
Algorithms Engineering
Data Pipeline
Edge Engineering

20.
Algorithms Engineering
R&D of algorithmic components for product

21.
Precompute
Online
Services
Model Trainer
New data available

22.
Infrastructure abstraction
Polyglot support & Local Dev Ex
Dynamic resource allocation &
distributed job scheduling
Requirements

23.
Model Trainer Online
Services
Precompute
New data available
Distributed scheduler
+

24.
Distributed Scheduling & Task Management
Model Trainer Spinnaker Other Apps
Fenzo

25.
Infrastructure abstraction
Polyglot + Local Dev
Distributed Scheduling
Num of Containers

26.
Docker @Netflix - Use Cases
Algorithms Engineering
Data Pipeline
Edge Engineering

27.
Data Pipeline
Publish, Collect, Process, Move Data

28.
Local Dev Ex
Infrastructure abstraction
Fine-grained resource management
Requirements

29.
Job

30.
Infrastructure abstraction
Local Dev Ex
Fine grained resource Management
Num of Containers

31.
Docker @Netflix - Use Cases
Algorithms Engineering
Data Pipeline
Edge Engineering

32.
Edge Engineering
Entry point to Netflix for all device traffic

33.
Automation & Tools

34.
Automation & Tools
Node.js
Html,
JS

35.
Infrastructure abstraction
Local dev, Polyglot - Java, Groovy,
Node.js
Fine-grained resource
management
Requirements

36.
Ops Tooling
Networking, IPC
Security
Insights
Failure Testing
Performance
And more…
Requirements++
Long running Services
vs.
Batch Jobs

37.
Ops Tooling
Networking, IPC
Security
Insights
Failure Testing
Performance
And more…
Requirements++
Long running Services
vs.
Batch Jobs

38.
Local Dev
Test/
Debug
Open PR
CI
Build/Test
AMI
Bakery
Deploy
Spinnaker
Dist Job
Sched

39.
Gradle docker plugin to generate Dockerfile
Spinnaker CloudProvider provides abstraction
Bakery support for image generation
Docker registry for internal images
libnetwork plugin for addressable IP
Building out the pieces

40.
Dev Ex Wins
Faster dev iterations
Run prod image locally; attach debugger
Multiple branch/config variations
Hot reloading of files
‘Roman Riding’ with AMIs

41.
ONNot There Yet!

42.
‘PaaS-like' Dev Experience
Production scale/Operational readiness
Optimized resource utilization

43.
Infrastructure Abstraction
Dev Ex
Production ready Online Services
Polyglot env, Packaging

44.
Thank You
@sangeetan

45.
The Netflix Algorithms Engineering & Data Pipeline
teams
Acknowledgements
Photo Creditshttps://goo.gl/sP5T4X
https://goo.gl/JaNlyf
https://goo.gl/rZXws1
http://goo.gl/V4UFmR
https://goo.gl/R08GPQ
http://tastyk.deviantart.com/art/Evolution-149719618
https://goo.gl/W2UKv0
https://goo.gl/fRcpPN
https://goo.gl/tF199H
https://goo.gl/K1OH9T wildflowers
https://goo.gl/KSaL5s roadmap
https://goo.gl/8JCDQy odometer
https://goo.gl/c4RuSR takeaway box

46.
Watch the video with slide synchronization on
InfoQ.com!
http://www.infoq.com/presentations/netflix-
containers