This document summarizes a presentation about Amazon EC2 Container Service (ECS). It covers the key topics of cluster management, container scheduling, container deployment, scaling ECS, logging and monitoring, and service discovery. For each topic, it provides an overview and links to additional resources with more in-depth information.

1. Daniele Stroppa
AWS Solutions Architect
@moviolone
Amazon EC2 Container Service:
Deep Dive

2. Agenda
• Cluster management
• Container scheduling
• Container deployment
• Scaling Amazon ECS
• Logging & Monitoring
• Service discovery

3. Cluster management

5. Cluster management with Amazon ECS
• Management of followers via ECS Agent
• Dispatching of sub-tasks to proper location
• Cluster state inspection
http://amzn.to/1jlHvnU

6. Cluster management under the hood
• Paxos-based transactional journal based data store
• Writes are committed as transaction in the journal with
order-based ID. The current value is the sum of all
transactions made as recorded by the journal.
• Reads are simply a snapshot in time of the journal. For a
write to succeed, the write proposed must be the latest
transaction since the last read.
http://bit.ly/1M9gGiv

7. Container scheduling

8. Amazon ECS Schedulers
• Task scheduler
• Run tasks once
• Batch jobs
• Service scheduler
• Load balancing
• Health management
• Scale-up and scale-down
• Update management
• AZ aware

9. Custom Schedulers
1. Calls the ECS List* and Describe* API operations to
determine the current state of the cluster.
2. Selects one (or more) container instances according to
the logic implemented.
3. Calls StartTask API to start a task on the selected
container instance.
http://amzn.to/1L9jmHT

10. Integration with Apache Mesos schedulers
• Allows you to use Apache Mesos schedulers like
Marathon and Chronos with ECS
• The ECSSchedulerDriver interprets the command given
when scheduling jobs with Mesos and starts a task with
TaskDefinition family:revision
http://bit.ly/1WXd6L3

11. Container deployment

12. CI/CD Partners

13. Continuous delivery with Jenkins
Build image
Push image
Update service

14. Continuous delivery with Jenkins
• Webhook to trigger job execution
• Docker Build and Publish plugin to build Docker image
• Amazon ECS API to update service
• Bonus – Cloudbees ECS plugin to run slave
http://amzn.to/1GbheTp

15. Amazon ECS CLI
• Easily create Amazon ECS clusters & supporting
resources such as EC2 instances
• Run Docker Compose configuration files on Amazon
ECS
• Available today – http://amzn.to/1jBf45a

16. Deploy Compose app with Amazon ECS CLI
> ecs-cli compose up
> ecs-cli compose ps
> ecs-cli compose service create
> ecs-cli compose service start
http://amzn.to/1jBf45a

17. Scaling Amazon ECS

18. Scaling with Amazon ECS CLI
> ecs-cli scale
> ecs-cli compose scale
> ecs-cli compose service scale
http://amzn.to/1jBf45a

19. Scaling with CloudWatch and AWS Lambda
http://amzn.to/1QbuG9Y

20. Logging & monitoring

21. Logging and monitoring on Amazon ECS
• Amazon CloudWatch Logs
• Amazon CloudTrail
• Partners & third party tools

22. Logging with Amazon CloudWatch Logs
• Centralized logs
• Collect, set up metrics, alarms
• View as graphs
• No need to modify your containers
• One additional container:
• rsyslog
• CloudWatch Logs agent

23. Logging Amazon ECS API with AWS CloudTrail
{
"eventVersion": "1.03",
"userIdentity": {…},
"eventTime": "2015-10-12T13:57:33Z",
"eventSource": "ecs.amazonaws.com",
"eventName": "CreateCluster",
"awsRegion": "eu-west-1",
"sourceIPAddress": "54.240.197.227",
"userAgent": "console.amazonaws.com",
"requestParameters": {
"clusterName": "ecs-cli"
},
http://amzn.to/1LjJT8N

24. Logging Amazon ECS API with AWS CloudTrail
"responseElements": {
"cluster": {
"clusterArn": "arn:aws:ecs:eu-west-
1:560846014933:cluster/ecs-cli",
"pendingTasksCount": 0,
"registeredContainerInstancesCount": 0,
"status": "ACTIVE",
"runningTasksCount": 0,
"clusterName": "ecs-cli",
"activeServicesCount": 0
}
},
[…]
http://amzn.to/1LjJT8N

25. Monitoring Amazon ECS with Datadog
http://bit.ly/1R723Lm

26. Monitoring Amazon ECS with Sysdig Cloud
http://bit.ly/1jrmvvD

27. Service discovery

28. Service discovery on Amazon ECS
• Amazon ECS Service Scheduler
• Amazon Route 53 private zone
• Partners & third party tools

29. Service Discovery with Amazon Route 53
http://bit.ly/1IjrCDj

30. Service Discovery with Weaveworks
http://bit.ly/1LkRjJ9

31. Service Discovery with Consul
http://amzn.to/1JZL5gz

32. Daniele Stroppa
AWS Solutions Architect
@moviolone
Thank you!