5. Issues with static partitionning
- Increased Complexity :
Machines/vm sprawl
Software failures gets harder to handle as
machines number grow
- Huge overheard, on development,
- Planning for failure and maintenance gets harder
- Not really developer friendly. Long cycles to
deployments.
#Devoxx #DV14 #mesos #docker
@samklr
7. Resources utilisation under static partitionning
Play/Akka Redis
#Devoxx #DV14 #mesos #docker
@samklr
Hadoop
Put together, we would get
8. Enters Google stage left ...
Google “Borg” .Unofficial name.
Moved to a new system called Omega
Bundles all the cluster’s machines into a single pooled
view of resources.
Evolved to Omega.
#Devoxx #DV14 #mesos #docker @samklr
10. Mesos : Level of indirection
1. Run multiple types of distributed systems on the same cluster of
machines, and dynamically share resources more eficiently.
2. Provide failure detection, task distribution, task tracking, task
monitoring, low level resources management, and fine grained
resources sharing. And scale to the thousands of nodes.
#Devoxx #DV14 #mesos #docker @samklr
15. #Devoxx #DV14 #mesos #docker
@samklr
Mesos Framework
A framework is an application that runs distributed applications on
Mesos.
It has two components :
The Scheduler : Interacts with Mesos, to book resources and
launch tasks within the mesos slaves.
Executors : gets information from the framework like
environmental variables,configuration information
and run tasks inside the mesos slaves.
16. Mesos Framework
A framework is an application that runs distributed applications on
Mesos.
It has two components :
The Scheduler : Interacts with Mesos, to book resources and
launch tasks within the mesos slaves.
Executors : run tasks in mesos slaves.
Mesos provides an API to build its own frameworks. And there are SDKs
in mulitple languages.
#Devoxx #DV14 #mesos #docker
@samklr
18. Resources allocation process
- Mesos allocate resources via
“resources offers”
- Resources are a snapshot of
available resources that a
scheduler will use to run
tasks on mesos slaves.
#Devoxx #DV14 #mesos #docker @samklr
29. Mesos uses Linux Containers for resource isolation.
Mesos containers are built around cgroups , to provide isolation of
CPU, memory, disk … and namespaces for network isolation.
Started leveraging containerirsation since 2010
#Devoxx #DV14 #mesos #docker
@samklr
37. Fault recovery & HA
- Mesos Master uses Zookeeper for leader election and leader
discovery.
- Mesos Master have a Registrar to log all the tasks and
slaves info, implemented as a replicated log using
MultiPaxos for consensus
#Devoxx #DV14 #mesos #docker
@samklr
38. Fault recovery & HA
- Mesos Master uses Zookeeper for leader election and leader
discovery.
- Mesos Master have a Registrar to log all the tasks and
slaves info, implemented as a replicated log using
MultiPaxos for consensus
- Mesos has a slave recovery mechanism. So user’s tasks keep
running whenever a slave proces goes under.
#Devoxx #DV14 #mesos #docker
@samklr
39. Fault recovery & HA
- Mesos Master uses Zookeeper for leader election and leader
discovery.
- Mesos Master have a Registrar to log all the tasks and
slaves info, implemented as a replicated log using
MultiPaxos for consensus
- Mesos has a slave recovery mechanism. So user’s tasks keep
running whenever a slave proces goes under.
Slaves checkpoint information to their local disks (task
info, status updates, etc ), they recover checkpointed data
and reconnect with old tasks when they restart.
#Devoxx #DV14 #mesos #docker
@samklr
41. Mesos Framework
A framework is an application that runs distributed applications on
Mesos.
It has two components :
The Scheduler : Interacts with Mesos, to book resources and
launch tasks within the mesos slaves.
Executors : run tasks in mesos slaves.
Mesos provides an API to build its own frameworks. And there are SDKs
in mulitple languages.
#Devoxx #DV14 #mesos #docker
@samklr
43. What is Marathon ?
- A generic mesos framework to run long running services (web apps,
etc …)
- A distributed Init.d for the cluster.
- Runs any linux binary without modification (Tomcat, Play, …)
- Cluster wide process supervisor
-Private PaaS
- Service discovery
- Provide a self service Rest API for deployment
- Authentification & SSL
- Placement constrainst (nodes, racks, etc. (Like with Spark) )
- Service discovery and load balancing via HAProxy
#Devoxx #DV14 #mesos #docker
@samklr
