You have heard how containers are great for running microservices, but running and managing large scale applications with microservices architectures is hard and often requires operating complex container management infrastructure. So what exactly is needed to get microservices to run in production at scale? In this session, we will explore the reasoning and concepts behind microservices and how containers simplify building microservices based applications, and we will walk through a number of patterns used by our customers to run their microservices platforms. We will also dive deep into some of the challenges of running microservices, such as load balancing, service discovery, and secrets management, and we’ll see how Amazon EC2 Container Service (ECS) can help address them. We will also demo how you can easily deploy complex microservices applications using Amazon ECS.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Nathaniel Slater, AWS Solution Architecture
April 2017
Deep Dive on
Microservices and Docker

2. What to Expect from the Session
• Microservices architecture
• Amazon ECS
• Task placement
• The twelve-factor app with Amazon ECS

3. Microservices Architecture

4. What are microservices?
“A software architecture style in which complex
applications are composed of small, independent
processes communicating with each other using
language-agnostic APIs. These services are small, highly
decoupled and focus on doing a small task, facilitating a
modular approach to system-building.” - Wikipedia
https://en.wikipedia.org/wiki/Microservices

5. Monolithic vs. Microservices
webserver
.package
Order UI
Order Service
Inventory
Service
Shipping
Service
OrderUI
Inventory
Service
Order
Service
Shipping
Service

6. Characteristics of Microservice Architectures
Do one
thing wellIndependent
Decentralized
Black box
Polyglot
You build it, you run it

7. Why Amazon ECS
• Fully managed, elastic service – you don’t
need to run anything, and the service
scales as your microservices architecture
grows
• Shared state optimistic scheduling
• Integration with Amazon CloudWatch for
monitoring and logging
• Integration with Code* services for
continuous integration and delivery (CI/CD)

8. Deploying Containers on ECS – Choose a Scheduler
Batch Jobs
ECS task scheduler
Run tasks once
Batch jobs
RunTask (random)
StartTask (placed)
Long-Running Apps
ECS service scheduler
Health management
Scale-up and scale-down
AZ aware
Grouped containers

9. WatchBot
ECS Cluster
SQS
Watcher
Container
Running
Tasks
14
Data Processing
Services
3500
Peak Container
Instances
500 million
Compute Hours
Used in 2016

10. Instacart’s PaaS
Coordinated through RDS database
Same tooling local & cloud
Jobs take desired state from DB, and turn
it into cloud reality

11. • Media Encoding Experimentation
• Digital Watermarking
Netflix

12. Amazon ECS: Under the Hood
ALB ALB
AZ 1 AZ 2
user / scheduler
Scheduler
Cluster State Service
Placement Engine
Event Stream

13. Task Placement Examples

14. g2.2xlarge t2.small t2.micro t2.medium
t2.medium t2.small g2.2xlarge
t2.small
t2.small t2.medium
us-east-1aus-east-1d
Placement: Targeting Instance Type & Zone

15. g2.2xlarge t2.small t2.micro t2.medium
t2.medium t2.small g2.2xlarge t2.small
us-east-1aus-east-1d
g2.2xlarge t2.medium
t2.micro t2.small
us-east-1c
Placement: Spread across Zone and Binpack

16. g2.2xlarge t2.small t2.micro t2.medium
t2.medium t2.small g2.2xlarge t2.small
us-east-1aus-east-1d
g2.2xlarge t2.medium
t2.micro t2.small
us-east-1c
Placement: Affinity and Anti-Affinity

17. Running a Service

18. t2.medium t2.small t2.small
us-east-1aus-east-1d
t2.medium t2.micro t2.small
us-east-1c
Placement: Multiple Services on a Cluster

19. t2.medium g2.2xlarge t2.micro t2.small
t2.small t2.small g2.2xlarge t2.small
t2.small t2.small
g2.2xlarge t2.small
Placement: Services – Distinct Instances

20. The Twelve-Factor App
with Amazon ECS
Reference Architectures

21. The Twelve-Factor App
1. Codebase
2. Dependencies
3. Config
4. Backing services
5. Build, release, run
6. Processes
7. Port binding
8. Concurrency
9. Disposability
10. Dev/prod parity
11. Logs
12. Admin processes
https://12factor.net/

22. Reference Architectures

23. Example Microservice Architecture on ECS
Amazon
ECR
Amazon
RDS
Application Load
Balancer
ECS Cluster
ECS Cluster
IAM
Amazon API
Gateway*
Amazon
Route 53
Amazon CloudWatch

24. Automatic Service Scaling
Publish metrics
Auto Scaling ECS service
Availability
Zone A
Availability
Zone B
TASK A
Add/Remove ECS
tasks
TASK C
TASK BScaling Policies
Amazon
CloudWatch
Amazon ECS
Application
Load Balancer

25. IAM Roles For Tasks
ECS Cluster
EC2 Instance EC2 Instance
TASK A
TASK B
TASK B
Amazon
DynamoDB
Amazon
S3

26. Continuous Deployment
AWS
CodeCommit AWS
CodePipeline
AWS
CodeBuild
Amazon
ECR
Amazon ECS
instance
Spot
Instance
AWS
CloudFormation
1. Commit
Code
2. Trigger
Pipeline
3.Build
Artifact
5.Update
Stack
6. Update Service
4. Push Image

27. Blue-Green Deployments
TaskTask
Route 53
record set
with
weighted
routing
policy
0%
100%

28. Secrets Management
•prod.app1.db-pass
•general.license-code
•prod.app2.user-name
ECS Cluster
EC2 Instance EC2 Instance
TASK A
TASK B
TASK B
EC2 System Manager –
Parameter Store

29. Service Discovery with Route 53 and Application
Load Balancers
Application Load
Balancer
i-aaa i-bbb i-ccc
i-aaa i-bbb i-ccc
oAuth Target Group
8080 8081
Portal Target Group
Weather Target Group
8000 8001 8002
8080
Amazon
Route 53
ECS Cluster
mydomain.com
mydomain.com mydomain.com/weather
mydomain.com/auth

30. Consuming Events for Service Discovery
app1-tst  10.1.0.11
db1-tst  10.1.0.14
app2  10.1.0.16
db2  10.1.0.18
my-app  10.1.0.20
websrv1 10.1.0.1
websrv2 10.1.0.2
websrv3 10.1.0.4
app-dev1 10.1.0.9
app-dev2 10.1.0.5
app-dev3 10.1.0.8
db-dev 10.1.0.19

31. Governance: AWS Service Catalog
Organizations Developers
Control
Standardization
Governance
Agility
Self-service
Time to market

32. Creates portfolio
Adds constraints and grant access
1
4
5
Administrator
Portfolio
Users
Browse Products
6Launch Products
Creates
product3Authors template2
Product
X
Product
Y
Product
Z
7
Deploys
stacks
Notifications
Notifications
8
8
Governance: AWS Service Catalog

33. Build Learn
Try It out
Deploy

34. Thank You

35. Introducing Blox

36. What is Blox?
Choice
Control
Developer Experience

37. Building with Blox
scheduler cluster state service

38. Set Up Blox Locally
Step 1: Run git clone https://github.com/blox/blox.git
Step 3: Run docker-compose up -d
Step 4: Start using Blox locally
Step 2: Deploy CloudFormation template to configure ECS event stream and SQS queue

39. Swagger Spec: Cluster State Service

40. Swagger Spec: Daemon Scheduler

41. Daemon Scheduler

42. t2.small t2.small t2.small
Blox: Daemon Scheduler
t2.small t2.small t2.small
scheduler cluster state service
t2.small t2.small t2.small

43. Step 1: Create an Environment

44. Step 2: Create a Deployment

45. Step 3: Add Container Instances

46. Step 4: Monitoring and Scaling a Cluster

47. Contributing to Blox
• Blox is licensed under Apache 2.0
• Open an issue or pull request
• Watch our roadmap on GitHub
• Check out our Gitter channel

48. The Blox Vision