This presentation was part of the talk "Re-Platforming All the Things" by Thomas Rampelberg (Product Manager at Mesosphere) at MesosCon 2015 Try Mesosphere for Free: https://mesosphere.com/try

1. Re-Platforming All the
Things

2. © 2015 Mesosphere, Inc.
Who in the world are you?
Thomas Rampelberg
Product Manager
thomas@mesosphere.io
@grampelberg
2

3. © 2015 Mesosphere, Inc.
Who do you work for?
3

4. © 2015 Mesosphere, Inc.
Interested in trying it out now?
• Mesosphere DCOS
• Apache Mesos + Marathon
• https://mesosphere.com/amazon/
4

5. © 2015 Mesosphere, Inc.
• The datacenter of yesterday
• The datacenter of today
• The datacenter of tomorrow
Agenda
5

6. © 2015 Mesosphere, Inc. 6
Yesterday

7. © 2015 Mesosphere, Inc.
History of the World
7
The Mainframe
Linux + x86
Virtual Machines / Cloud
Containers

8. © 2015 Mesosphere, Inc.
Mainframes
• Applications - 1 service
• Hardware - 1 box
• Redundancy - Systems Architecture
• Reboot - Never
• Services - 100s/”host”
8

9. © 2015 Mesosphere, Inc.
• Applications - 2(ish) services
• Hardware - 10s of boxes
• Redundancy - Multiple boxes
• Reboot - Occasionally
• Services - 10s/”host”
Linux + x86 - Commoditization
9

10. © 2015 Mesosphere, Inc.
Virtual Machines / Cloud
• Applications - 3+ service
• Hardware - 100s of “hosts”
• Redundancy - Hardware + Software
• Reboot - Occasionally
• Services - A couple/”host”
10

11. © 2015 Mesosphere, Inc.
• Applications - Microservices
• Hardware - 1000s of containers
• Redundancy - Multiple boxes
• Reboot - Often
• Services - 1 per container
Containers
11

12. © 2015 Mesosphere, Inc. 12
Today

13. © 2015 Mesosphere, Inc.
Pets
• They’re Individuals
• You have to clean up
• They fail eventually
13

14. © 2015 Mesosphere, Inc.
• Treated as a herd
• Cow pies happen
• They fail eventually
Cattle
14

15. © 2015 Mesosphere, Inc.
How are these different?
• 144 Cores
• 1.15 TB Memory
15
• 144 Cores
• 1.15 TB Memory

16. © 2015 Mesosphere, Inc.
“Sixty-four cores or 128 cores in a single computer looks a lot like 64 or 128
machines in a datcenter”- Benjamin Hindman
16

17. © 2015 Mesosphere, Inc.
One Big Computer
• Manage Resources Globally
• Manage Failures Globally
• Scale Across Hosts
17

18. © 2015 Mesosphere, Inc.
• Scalability
• Multi-tennancy
• Efficiency
• Reliability & availability
• On-demand resources
What do you get?
18

19. © 2015 Mesosphere, Inc.
Who’s doing it?
Closed Source
• Google Borg
• Google Omega
19

20. © 2015 Mesosphere, Inc.
Who’s doing it?
20
Open Source
• Apache Mesos
• Apache Hadoop (YARN)
• Kubernetes

21. © 2015 Mesosphere, Inc.
Apache Mesos
21

22. © 2015 Mesosphere, Inc.
Overview
• Borg is awesome, let’s do something
better
• Came out of UC Berkeley's AMP Lab
• First paper in 2009
• Running at Twitter and Apple
22

23. © 2015 Mesosphere, Inc.
• Separate resources from placement
• Focus on running batch jobs and
services
• All about the scheduler
What’s Different?
23

24. © 2015 Mesosphere, Inc.
Apache Hadoop (YARN)
24

25. © 2015 Mesosphere, Inc.
Overview
• Hadoop needed a better scheduler
• YARN became the foundation for
Hadoop 2.0
• Paper came out in 2013
• Running at Yahoo and LinkedIn
25

26. © 2015 Mesosphere, Inc.
• Centralize the resource and placement
decisions
• Optimize for somewhat long batch jobs
(eg. MR)
• Bake the storage layer into the solution
What’s Different?
26

27. © 2015 Mesosphere, Inc.
Kubernetes
27

28. © 2015 Mesosphere, Inc.
Overview
• Google took everything they learned
from Borg and Omega
• Kubernetes was open source in 2014,
just went 1.0 last month
• Running in production ….. ?
28

29. © 2015 Mesosphere, Inc.
• Containers first, only containers
• Pods are a first class citizen
• Labels, labels everywhere
What’s Different?
29

30. © 2015 Mesosphere, Inc. 30
Tomorrow

31. © 2015 Mesosphere, Inc.
Stateful Applications
• Persistent Volumes
• Durable Pods
• Distributed File Systems
31

32. © 2015 Mesosphere, Inc.
• Networking virtualization
• Flannel
• Weave
• Calico
• Dynamic load balancing
• IPv6
Service Discovery
32

33. © 2015 Mesosphere, Inc.
Security
• Not everyone is a small startup
• Compliance is important
• Long tail of process
• Containers are still early
• Schedule policy, automatically
33

34. © 2015 Mesosphere, Inc.
• Sizing is hard
• Not all processes need guarantees
• Let the scheduler sort it out
Oversubscription
34

35. © 2015 Mesosphere, Inc. 35
Next Steps

36. © 2015 Mesosphere, Inc.
Getting Started
• Apache Mesos - http://mesosphere.
com/product
• Apache Hadoop (YARN) - http://bit.
ly/1K13E4S
• Kubernetes - http://kubernetes.
io/gettingstarted/
36

37. © 2015 Mesosphere, Inc.
• http://chat.mesosphere.com/
• http://mesos.apache.org/community/
• https://wiki.apache.
org/hadoop/HowToContribute
• http://kubernetes.io/community/
Get Involved!
37

38. © 2015 Mesosphere, Inc.
Get Awesome
38