Managing and scaling hundreds of containers is a challenging task. A container management solution takes care of these challenges for you, allowing you to focus on developing your application. In this session, we cover the role and tasks of a container management solution and we analyze how four common container management solutions - Amazon EC2 Container Service, Docker for AWS, Kubernetes, and Apache Mesos - stack against each other. We also see how you can easily get started with each of these solutions on AWS.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Brandon Chavis
December 2, 2016
CON303
Introduction to Container
Management on AWS

2. Containers on AWS
Hypervisor Containers

3. Server
Guest OS
Bins/Libs
Bins/Lib
s
App2App1

4. Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS

7. “Okay, let’s run our containers on AWS.”
Choose your own adventure!Start!
You ==
Wizard?
Legacy
Apps?
Big
Data?
Cluster >
10000?
Love
Docker
CLI?
Cluster >
1000?
Cluster >
200?
Huge IT
team?
No
Yes
No
Yes
Inspiration:
http://adrianotto.com/wp-
content/uploads/2016/02/ch
oose-adventure-
1024x574.png
No Yes
Yes
YesNoYesNoYes
Yes

8. AWS Container Ecosystem
• Foundation
• Monitoring
• CI/CD
• Security
• PaaS

9. AWS Customers Have Options
Today we’ll cover:
• Amazon ECS
• CoreOS Tectonic (Kubernetes)
• Mesosphere DC/OS (Mesos)
• Docker Datacenter

10. Container Orchestration
Instance Instance Instance
OS OS OS
Container Runtime Container Runtime Container Runtime
App Service App App Service Service
Container Orchestration

11. Container Orchestration
myJob: {
Cpu: 10
Mem: 256
}
Orchestrator
Schedule
Run “myJob”

12. Container Orchestration
Instance/OS Instance/OS Instance/OS
App Service App App Service Service
Service Management
Scheduling
Resource Management
OrchestrationService Management
Availability
Lifecycle
Discovery

13. Container Orchestration
Instance/OS Instance/OS Instance/OS
App Service App App Service Service
Service Management
Scheduling
Resource Management
Orchestration
Scheduling
Placement
Scaling
Upgrades
Rollbacks

14. Container Orchestration
Instance/OS Instance/OS Instance/OS
App Service App App Service Service
Service Management
Scheduling
Resource Management
Orchestration
Resource
Management
 Memory
 CPU
 Ports

15. Schedulers
Cluster
Machines
Cluster State
Information
Monolothic Two-Level Shared State
No Concurrency Pessimistic
Concurrency
(offers)
Optimistic
Concurrency
(transactions)
Scheduling Logic

16. Kubernetes
Replication
Controller
API Server
Kubernetes Master
Kubelet KubeProxy Docker
Container
Containe
r
Pod Pod
Kubelet KubeProxy Docker
Containe
r
Containe
r
Pod Pod
Kubernetes Cluster
etcd

17. Kubernetes
Container orchestration or the OS for distributed compute
Upstream Kubernetes
● Container orchestration
● Horizontal scale
● High availability
● Service discovery & load
balancing
● Basic command line
operations
kubelet
default addons
(DNS)
controller manager
proxy
api server
scheduler
etcd
system kubelet
OS
Docker
Rkt
flannel
Kubernetes
Storage & Compute
apps/container/microservices
OS

18. Tectonic Extends
Upstream Kubernetes
● Installer
● Management console
● Painless updates
● Cluster scaling
● Disaster recovery
● Alerts and logging
● Security (integrated)
● Integration across
environments
Extending Kubernetes for the Enterprise
Security Mgmt
Kubernetes
CoreOS Linux
Cloud Integration
Update
Container Registry
Storage & Compute
apps/container/microservices

19. Tectonic
Kubernetes Management
● Single console across
environments
● AWS Auto Scaling
● Monitoring and logging
● Audit log
● Alert configuration
● View compute usage over
time and isolate by
namespace/cluster
Extending Kubernetes for the Enterprise
Security Mgmt
Kubernetes
CoreOS Linux
Cloud Integration
Update
Container Registry
Storage & Compute
apps/container/microservices

20. Tectonic
Operating System
● Packaged with CoreOS
Linux, tested against
Docker and rkt
● Rolling upgrades of OS
Extending Kubernetes for the Enterprise
Security Mgmt
Kubernetes
CoreOS Linux
Cloud Integration
Update
Container Registry
Storage & Compute
apps/container/microservices

21. Highlighted AWS Features
• Production-ready installs: Built from our learnings from kube-aws,
the Tectonic AWS installer installs a highly available self-hosted
Kubernetes cluster in your environment
• Managed upgrades on AWS: CoreOS applies CoreUpdate
functionality to Kubernetes.
• Disaster recovery: Managed backups and restore of Kubernetes
clusters via etcd to s3

22. Kube-AWS
• github.com/coreos/kube-aws
• kube-aws is a templating engine for AWS Cloudformation templates
• All assets (CloudFormation, userdata for instances) are declarative
templates that can be checked into git and version controlled
• Uses Amazon KMS to encrypt all secrets before putting them into assets
files. Secrets are unlocked once the machines boot into their IAM roles
• Spreading of workers and control plane nodes across AZs

23. Customer Use Case
Planet Labs launches and manages a fleet
of earth-imageing sattelites
Manages 100 million images and supports
tens of terabytes of data every day
Building a whole-earth dataset

24. Before:
• Operational overhead from
30 million jobs and 15,000
instances
• Challenges with
inconsistent packaging
and deployments across
teams
After:
• Kubernetes + CoreOS on
AWS
• Containers allowed
consistent app packaging
• Less rigid dependencies
• Less focus on infrastructure
Customer Use Case

25. Use Tectonic if:
• You want a Kubernetes experience tailored for
AWS
• You want to purchase support for Kubernetes
• You may want to consider another container
runtime (i.e. rkt)
Kube-aws:

26. Docker Datacenter (DDC)
Docker Universal Control Plane
Integrated
Security
Docker Engine
Container runtime, orchestration, networking, volumes, plugins
Docker Trusted Registry
Operating
Systems Config Mgt Monitoring LoggingCI/CD ..more..Images Networking Volumes
VirtualizationPublic Cloud Physical

27. Docker Datacenter
Universal Control Plane:
• Cluster manager
• Etcd for state management
• Swarm for scheduling
• LDAP integration, service
discovery, load balancing
built in
Docker Trusted Registry
• Private repository
• Image Signing
• LDAP/RBAC support
• S3 backend support

28. Docker Datacenter (DDC)
UCP
Manager
UCP
Manager
UCP
Manager
Internal distributed store
DTR
Replica
Worker
Raft consensus group
Image Registry
BYO TCP Load Balancer
UCP
Worker
UCP
Worker
UCP
Worker
UCP
Worker
push / pull
Admin / User
Deploy / manage
LDAP/AD
Monitoring
Logging
External
CA
Image
Storage
DTR
Replica
Worker
DTR
Replica
Worker

29. • Swarm Mode (Docker 1.12) Support
• Point and click UI to manage nodes,
services, containers, and networks
• Highly Available ( 3, 5, or 7 Controllers)
• Secure access control with LDAP/AD
support and granular RBAC
• Docker Content Trust : Image Signing
and Runtime Enforcement
UCP Manager
CS Docker Engine
Swarm Mode Manager Node
Monitoring
Web UI
Log
Aggregator
Access
Control
Auth Server
Docker Swarm
Docker Datacenter: UCP

30. DDC Quickstart – AWS Architecture
Availability Zone Availability Zone
Private subnet
Public subnet Public subnet
Private subnet
EC2
instances
EC2
instances
Elastic Load
Balancing
Elastic Load
Balancing
Internet
gatewayVPC NAT
gateway
VPC NAT
gateway
Internet
gateway

31. DDC Quickstart – AWS Architecture
Availability Zone Availability Zone
Private subnet Private subnet
Swarm NodesSwarm Nodes
DTR
Controller
DTR
Controller
DTR
Controller
UCP
Controller
UCP
Controller
UCP
Controller

32. Docker Datacenter:
Customer Use Case
ADP leverages Docker Datacenter on AWS to deliver Security And Scale For Both Legacy
And Microservices Applications
Challenge
Refactor legacy monolithic applications to microservices
Disparate systems at scale - hundreds of products, hundreds of thousands of clients
Solution
Leverages Docker Datacenter on AWS - UCP, DTR, CS Engine
Benefits
Adopt hybrid strategy – mix of big and small containers for any application creating an
evolutionary path forward to microservices on the Cloud.
Swarm spans across public and private infrastructure and across applications, allowing the
swarm to the abstraction layer between physical compute and the application teams.
Apps running in hardened containers, image signing, and multiple DTRs ensure a secure
environment
“Docker’s CaaS approach will enable us to drive transformation across the entire application
lifecycle from development to operations. With Docker, we will be able to ensure application
portability, whether it is between dev and ops or between the datacenter and the cloud.”

33. Use Docker Datacenter if…
• The “Docker Native” stack is important to you
• Your development workflow is built around Compose and the
Docker CLI
• You want Commercial Support for the Docker Engine
DDC Quickstart:

34. Mesos
Master
Marathon
ZooKeeper
Mesos + Marathon
Mesos Slaves
Long Running
Tasks
Jobs
Coordination &
Configuration

35. ● Collection of services to power your apps
● Service installation and lifecycle management
36
DC/OS Universe
Mesosphere Enterprise DC/OS
Large install base on premise and in the cloud
● Container & big data operations
● Security, fault tolerance & high availability
● Open core & production proven at scale
Container orchestration
Datacenter aggregation
Turnkey lifecycle mgt
User Interface & CLI
Security
Advanced
Networking
Monitoring &
Operations
Compliance
Elastic
● Supports hybrid deployments
Mesosphere DC/OS

36. Mesosphere DC/OS
Big Data
Analytics
Stateful
Service
Big Data and Stateful Apps
Mesosphere Enterprise DC/OS
Container
App
Container
App
CaaS PaaS
● Uses 66% less infrastructure
● Simplified operations
● Turnkey install of datacenter-wide services
● Easier to experiment with new tech (e.g., Spark)

37. Mesosphere DC/OS

38. Mesosphere DC/OS:
Customer Use Case
• Production event handling service
• Deployed to 3 AWS Regions
• 40 seconds to deploy new builds
• Infrastructure Density: 66% less instances
• Cost Reduction: 57% less spend

39. Use DC/OS If….
• You prefer technology that has been around longer than your
toddler
• You want to take advantage of the “DC/OS Universe”
• You want to also manage big-data applications with your
orchestration tool
DC/OS on AWS:

40. Docker
Task
Container Instance
Amazon
ECS
Container
ECS Agent
ELB
Internet
ELB
User /
Scheduler
API
Cluster Management Engine
Task
Container
Docker
Task
Container Instance
Container
ECS Agent
Task
Container
Docker
Task
Container Instance
Container
ECS Agent
Task
Container
AZ 1 AZ 2
Key/Value Store
Agent Communication Service
Amazon ECS

41. Amazon ECS
Container Management
at Any Scale
Flexible Container
Placement
Integration
with the AWS Platform

42. Components of ECS
Task
Actual containers running on
Instances
Task Definition
Definition of containers and
environment for task
Cluster
Fleet of EC2 instances on
which tasks run
Manager
Manage cluster resource
and state of tasks
Scheduler
Place tasks considering
cluster status
Agent
Coordinate EC2 instances
and Manager

43. Cluster, Scheduler, Task Scheduler
ManagerCluster
Task Definition
Task
Agent

44. Expedia
One of the world’s leading travel companies
• Primer – Internal deployment tool
• Supports various applications
• Creates GitHub repository, pipeline,
and monitoring by one click
• Based on ECS Optimized AMI,
configured by AWS
CloudFormation
• Zero-Downtime Instance
Replacement
http://www.slideshare.net/AmazonWebServices/deep-dive-on-microservices-and-amazon-ecs-64033400
Continuous Delivery to ECS with Primer
ECS Production Clusters – Serving 200 applications
14 instances: 56 apps (+ 19 canaries) 17 instances: 78 apps (+ 25
canaries)
35 instances: 107 apps (+ 23 canaries) 5 instances: 7 apps (+ 4 canaries)
Charts produced with c3vis: github.com/ExpediaDotCom/c3vis

45. Amazon Personalization
Distributed neural network learning on multiple GPUs
• From Apache Spark, run CPU
and GPU tasks transparently
• CPU: Amazon EMR
• GPU: Amazon ECS
• Package GPU libraries with
Docker image
• DSSTNE runs workloads in
parallel across hundreds of
GPUs
https://blogs.aws.amazon.com/bigdata/post/TxGEL8IJ0CAXTK/Generating-Recommendations-at-Amazon-Scale-with-Apache-Spark-and-Amazon-DSSTNE

46. Use ECS if:
• You want a managed service that scales with you
• You want to leverage native AWS integrations: IAM Roles,
ALB, VPC, Cloudwatch Logs, Autoscaling, ECR
• You want to build around AWS tools:
API/CLI/SDK/CloudFormation
• You want to leverage the strong ECS partner ecosystem

47. Conclusion:
• AWS has a rich ecosystem and supports every
major orchestration framework
• Whether you choose to use Amazon ECS or an
AWS Partner solution, our goal to provide the best
experience possible
• Container workloads: Think AWS

48. Thank you!