This presentation was made by Madhusudan Shekar of AWS as as main session in Container Conference 2018 on 3rd August in Bangalore. The Evolution to Microservices – A Container Story "Technology improves at a rapid pace across compute, storage and networking, but we tend to continue to build applications that are yet to take advantage of much of these capabilities. Cloud platforms provide a way to abstract these complexities from the developer, enabling them to adopt these innovations rapidly. In this session, we will look at how application architectures are evolving, and how it is now possible for developers to prioritise on innovation rather than operation and bring value to the user base."

1. Madhusudan Shekar
Principal Technical Evangelist @ Amazon
Monoliths to Microservices
@madhushekar23

2. “To finish the moment, to find the journey’s end in every step of
the road, to live the greatest number of good hours, is wisdom.”
— Ralph Waldo Emerson

3. Which journey are we on?
Data center Cloud

4. Which journey are we on?
Virtual Machines Containers Serverless
Data center Cloud

5. Evolution of Business Logic
Monolith Microservices Functions

6. Splitting
Monoliths
Ten Years Ago

7. Splitting
Monoliths
Ten Years Ago
XML & SOAP

8. Splitting
Monoliths
TenFiveYears Ago

9. REST JSON
Fast binary
encodingsSplitting
Monoliths
Five Years Ago

10. Splitting
Monoliths
TenFive Years Ago

14. Microservices
Five Years Ago

16. Microservices
Five Years Agoto Functions
Amazon
Kinesis
Amazon API
Gateway
Amazon SNS
Amazon S3
Amazon
DynamoDB
Amazon
SQS
Standard building brick
services provide standardized
platform capabilities

17. Amazon SNS
Amazon S3
Amazon API
Gateway
Amazon
SQS
Amazon
Kinesis
Amazon
DynamoDB
Microservices
to Functions
Business Logic
Glue between
the bricks
Standard building brick
services provide standardized
platform capabilities

18. Amazon SNS
Amazon S3
Amazon API
Gateway
Amazon
SQS
Amazon
Kinesis
Amazon
DynamoDB
Microservices
to Functions

19. Amazon SNS
Amazon S3
Amazon API
Gateway
Amazon
SQS
Amazon
Kinesis
Amazon
DynamoDB
Microservices
to Functions

20. Amazon SNS
Amazon S3
Amazon API
Gateway
Amazon
SQS
Amazon
Kinesis
Amazon
DynamoDB
Microservices
to FunctionsEphemeral

21. Microservices
to
Functions
Ephemeral

22. Microservices
to
Amazon API
Gateway
Amazon
SQS
Functions
Ephemeral

23. Microservices
to
Amazon API
Gateway
Amazon
Kinesis
Amazon
DynamoDB
Functions
Ephemeral

24. Microservices
to
Amazon API
Gateway
Amazon SNS
Amazon S3
Functions
Ephemeral

25. Amazon SNS
Amazon S3
Amazon API
Gateway
Amazon
SQS
Amazon
Kinesis
Amazon
DynamoDB
Microservices
to
Functions
Ephemeral
When the system is
idle, it shuts down and
costs nothing to run

26. Evolution of Business Logic
Monolith Microservices Functions

28. = 50 million deployments a year
Thousands of teams
× Microservice architecture
× Continuous delivery
× Multiple environments
5708 per hour – one every 0.63 seconds!

29. Which journey are we on?
Virtual Machines Containers Serverless
Data center Cloud

30. Containers

31. Portability
Flexibility
Efficiency
Speed
Same immutable images. Run anywhere.
Create modular environment. Decompose Apps.
Speeds up build and release cycles.
Optimize resource allocation.
Agility.

32. Microservices And Containers
• Do one thing, really well
• Any app, any language
• Test and deploy same artifact
• Self-contained services
• Isolated execution environment
• Faster startup
• Scaling and upgrading

33. ”Service oriented architecture composed of loosely coupled
elements that have bounded contexts.”
— Adrian Cockcroft

34. How do I run apps on many machines?

35. Running Containers
EC2 Instance EC2 InstanceEC2 InstanceEC2 InstanceEC2 Instance

36. Running Containers at Scale
Scheduling and Orchestration
Cluster Manager Placement Engine
Availability Zone #1 Availability Zone #2 Availability Zone #3

37. Amazon
RDS
ElastiCache
(Redis)
Amazon
S3
Amazon
SQS
Auto Scaling Group
Multi-AZ
Auto Scaling Group
Multi-AZ
Microservice 1 Microservice 2
Amazon ECS
McD API Middleware
Third-party
Delivery Platforms
Menu and
Restaurant
Master
Restaurant
Application Load Balancer

38. “Could you tell me which rack all my EC2 instances landed in.”
— (Almost) no one

39. Enter Container Orchestration

40. Pick your orchestration tool
Pick your compute engine

41. EC2 INSTANCES
ECS
AGENT
TASK
Containers
TASK
ECS
AGENT
TASK TASK
AGENT COMMUNICATION
SERVICE
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK TASK
Internet LOAD
BALANCER
LOAD
BALANCER
Containers
Containers Containers
Containers Containers
Amazon ECS

42. AWS VPC with CNI
networking
Advanced task
placement
Deep integration
with AWS platform
ECS CLI…{ }
Global footprint
Powerful scheduling
engines
Auto scaling
CloudWatch metrics
Load balancers
PRODUCTION WORKLOADS ON AWS

43. CUSTOMERS RUNNING CONTAINERS WITH
ECS

44. WHAT ABOUT KUBENETES?
“According to a recent survey, 63% of
users were already hosting Kubernetes
on Amazon EC2. This was up from 44
percent a year ago.”
https://www.cncf.io/blog/2017/06/28/survey-shows-kubernetes-leading-orchestration-platform/

45. AWS and CNCF
Promote Cloud Native to enterprise
customers
Integrate CNCF components into
AWS ECS – CNI, containered, etc.
Integrate Kubernetes with AWS –
installers, IAM, security, EKS, etc.
CNCF serverless working group
Blog post
medium.com/@adrianco
Kubernetes scalability testing will be
moving to AWS, supported by
credits to CNCF

46. Code
Improve
AWS was a founding member of
containerd – docker runtime
CNCF Contributions
Kubernetes work on installers,
security, and networking
integration with AWS
CNI – Container Networking
Interface extended and
integrated into ECS and EKS

47. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Native VPC networking
with CNI plugin
Pods have the same VPC
address inside the pod
as on the VPC
Simple, secure networking
Open source and
on Github
…{ }

48. Nginx Pod
Java Pod
ENI
Secondary IPs:
10.0.0.1
10.0.0.2
Veth IP: 10.0.0.1
Veth IP: 10.0.0.2
Nginx Pod
Java Pod
ENI
Veth IP: 10.0.0.20
Veth IP: 10.0.0.22
Secondary IPs:
10.0.0.20
10.0.0.22
ec2.associateaddress()
VPC Subnet – 10.0.0.0/24
Instance 1 Instance 2

49. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CNI plugin
Allow Kubernetes users to take advantage of native
VPC networking in their Kubernetes pods

50. Manage d Kube r ne te s on AWS
Amazon Elastic Container Service for
Kubenetes (EKS)
Hybrid cloud
compatible
Highly available Automated
upgrades and
patches
Integrated with
AWS Services
CloudTrail, CloudWatch,
ELB, IAM, VPC,
PrivateLink
NEW!

51. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Kubectl
3) Authorizes AWS Identity with RBAC
K8s API
1) Passes AWS Identity
2) Verifies AWS Identity
4) K8s action
allowed/denied
AWS Auth

52. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

53. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

54. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

55. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

56. Are clusters really cloud native?

57. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
INTRODUCING FARGATE

58. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CHANGING COMPUTE
CONSUMPTION MODEL
No instances
to manage
Task
native API
Resource
based pricing
Simple, easy to use,
powerful – and new
consumption model

59. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scheduling and Orchestration
Cluster Manager Placement Engine
ECS
AMI
Docker
agent
ECS
agent
EC2 Instance
ECS
AMI
Docker
agent
ECS
agent
EC2 Instance
ECS
AMI
Docker
agent
ECS
agent
EC2 Instance

60. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Pick your orchestration tool
Pick your compute engine

61. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Custom code and Services
Lots of choices of frameworks and API
mechanisms
Where needed optimize serverless
applications by also building services
using containers to solve for
- Lower startup latency
- Long running compute jobs
- Predictable high traffic
Serverless events and functions
Standardized choices
Combine building blocks to achieve
outcome
- AWS Lambda
- API Gateway
- Amazon SNS, SQS
- Amazon DynamoDB
- AWS Step Functions
S e r v e r l e s sC o n t a i n e r s
Which for Microservices ?

62. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you
madsheka@amazon.com
@madhushekar23