The AWS Cloud offers infrastructure resources optimized for running containers, as well as a set of orchestration services that make it easy for you to build and run containerized applications in production. In this session we will review how Docker containers are used to build microservices and how Amazon Elastic Container Service (Amazon ECS) and AWS Fargate are used for container orchestration to help customers like FINRA run and scale containerized applications on AWS.

1. P U B L I C S E C T O R
S U M M I T
WASHINGTON, DC

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Running Containers in a Hybrid Environment
Harsha Nippani
Solutions Architect
Amazon Web Services
2 9 9 9 4 6
Latha Nagaraj
Principal Application Architect
FINRA

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Agenda
• Containers and Docker
• Amazon Elastic Container Registry (Amazon ECR)
• Amazon Elastic Container Service (Amazon ECS)
• AWS Fargate
• Customer Use Case - FINRA

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
The Problem
• Different application stacks
• Different hardware deployment
environments
• How to run all applications
across different environments?
• How to easily migrate from one
environment to another?
Static
website
Web
fronted
Background
workers
User DB
Analytics
DB
Queue
Dev VM
QA
Server
Single
Prod
Server
Onsite
Cluster
Public Cloud
Contributor
’s laptop
Customer
Servers

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
The Solution
• Unit of software delivery
• Lightweight, portable, consistent
• Deploy and run everywhere
• Deploy and run anything
Static
website
Web
fronted
Background
workers
User DB
Analytics
DB
Queue
Dev VM
QA
Server
Single
Prod
Server
Onsite
Cluster
Public Cloud
Contributor
’s laptop
Customer
Servers

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Docker container image
bootfs
kernel
Base image
Image
Image
Container
References
parent
image
Read only image that is used
as a template to launch a
container.
Start from base images that
have your dependencies, add
your custom code.
Docker file for easy,
reproducible builds.

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Virtual Machines Vs Containers
Server (Host)
Host OS
Hypervisor
App 2
Guest OS Guest OS Guest OS
Bins/Libs Bins/Libs Bins/Libs
App 1
App 2
App 3
VM
Server (Host)
Host OS
Docker
Bins/Libs Bins/Libs Bins/Libs
App 1 App 2 App 3
Container

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Docker Images
Packaged
Application Code
Reproducible Immutable
Portable

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
AWS Container Services Landscape
MANAGEMENT
Deployment, Scheduling,
Scaling & Management of
containerized applications
HOSTING
Where the containers run
Amazon ECS
Amazon Elastic
Container
Service for
Kubernetes
(Amazon EKS)
Amazon
EC2
AWS
Fargate
IMAGE REGISTRY
Container Image Repository
Amazon ECR

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Amazon Elastic Container Registry
Fully Managed
* Tight Integration with Amazon ECS
* Integration with Docker Toolset
* Management Console and AWS
Command Line Interface (AWS CLI)
Highly Available
* Amazon Simple
Storage Service
(Amazon S3) backed*
Regional endpoints
Secure
* AWS Identity and Access Management
(IAM) Resource-based Policies
* AWS CloudTrail Audit Logs
* Images encrypted at transit and at rest

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Evolution of Compute on AWS
AWS Cloud AWS Cloud AWS Cloud
EC2
Instance
Amazon EC2 Amazon ECS
Containers
in the cloud
(no servers to manage)
Container
EC2 Instance
 
Container
VMs in the cloud
Containers in
Amazon EC2
instances
in the cloud
AWS Fargate
(EC2 launch type)

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Amazon EC2
Instance
TaskTask
Task Task
Amazon EC2
Instance
TaskTask
Task Task
Amazon EC2
Instance
TaskTask
Task Task
Amazon EC2
Instance
TaskTask
Task Task
Amazon EC2
Instance
TaskTask
Task Task
RUNNING CONTAINERS

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
ECS Service
Amazon ECS—Task & Service
AMAZON EC2 INSTANCES
LOAD
BALANCER
Internet
ECS
AGENT
TASK
Container
TASK
Container
ECS
AGENT
TASK
Container
TASK
Container
AGENT COMMUNICATION
SERVICE
Amazon ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK
Container
TASK
Container
LOAD
BALANCER

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Define application containers: Image
URL, CPU & Memory requirements,
etc.
register
Task Definition
create
Cluster
• Infrastructure Isolation
boundary
• IAM Permissions boundary
run
Task
• A running instantiation of a
task definition
create
Service
Elastic Load
Balancer (ELB)
• Maintain n running copies
• Integrated with ELB
• Unhealthy tasks
automatically replaced
CONSTRUCTS

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Scheduling and Orchestration
Cluster Manager Placement Engine
RUNNING CONTAINERS AT SCALE WITH
ECS
Availability Zone #1 Availability Zone #2 Availability Zone #3

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
PRODUCTION WORKLOADS ON ECS
Amazon Virtual
Private Cloud
(Amazon VPC)
networking mode
Advanced task
placement
Deep integration
with AWS services
Amazon ECS AWS
CLI
…{ }
Global footprint
Powerful scheduling
engines
Auto scaling
Amazon CloudWatch
metrics
Load balancers

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Scheduling and Orchestration
Cluster Manager Placement Engine
ECS
AMI
Docker
agent
ECS
agent
Amazon EC2 Instance
ECS
AMI
Docker
agent
ECS
agent
Amazon EC2 Instance
ECS
AMI
Docker
agent
ECS
agent
Amazon EC2 Instance
ECS and Fargate

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
No clust er s
t o manage
M anages under lying
inf r ast r uctur e
Easy t o r un,
easy t o scale
AWS FARGATE

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Define application containers:
Image URL, CPU & Memory
requirements, etc.
register
Task Definition
Create
Cluster
• Infrastructure Isolation
boundary
• IAM Permissions boundary
run
Task
• A running instantiation
of a task definition
• Use AWS FARGATE
launch type
create
Service
Elastic
Load
Balancer
• Maintain N running
copies
• Integrated with ELB
• Unhealthy tasks
automatically replaced
FARGATE CONSTRUCTS

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Bridge networking mode
Any non-local traffic to/from the containers must transit through eth0
Traffic between containers within the same task transit through docker-
0
docker-0 bridge
eth0
network
interface
default network namespace
c1 namespace c2 namespace
task
Amazon EC2
Instance
c1 c2

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
awsvpc networking mode
Containers within the task are in the same namespace can talk over local host
All traffic transits through the task Elastic Network Interface (ENI). No
dependence on host’s eth0
eth0
network
interface
default namespace
task namespace
EC2 Instance
c1 c2
task
task
network
interface
local
host

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
VPC INTEGRATION
172.31.0.0/16
Subnet
172.31.1.0/24
Internet
Other Entities in VPC
EC2 LB DB etc.
Private IP
172.31.1.164
Launch your Fargate Tasks into subnets
Under the hood :
• We create an ENI
• The ENI is allocated a private IP from your
subnet
• The ENI is attached to your task
• Your task now has a private IP from your
subnet!
You can assign public IPs to your tasks
Configure security groups to control inbound &
outbound traffic
ENI Fargate
TaskPublic /
208.57.73.13 /

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Policy
Before Task Roles
Tasks rely on the Instance Role for permissions
Instance Role must have the superset of all permissions. Violates principle of least privilege.
EC2 Instance Role
...
{
"Action": [”ddb:*”],
"Effect": "Allow",
}
...
{
"Action": [”s3:*”],
"Effect": "Allow",
}
task 1
AWS SDK/CLI
task 2
AWS SDK/CLI

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Task Roles
No reliance on Instance Role for permissions
EC2 Instance RoleTask 1 Policy
task 1
AWS SDK/CLI
task 2
AWS SDK/CLI
{
"Action": [”ddb:*”],
"Effect": "Allow",
} Task1 Role
Task 2 Policy
{
"Action": [”s3:*”],
"Effect": "Allow",
}Task2 Role

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Architecture Reference: Private Task Setup
Public subnet Private subnet
Fargate
TaskENI
Private IP
172.31.1.164
NAT Gateway
Public EIP
34.214.162.237
Internet
Gateway
172.31.0.0/16
172.31.2.0/24 172.31.1.0/24
Destination Target
172.31.0.0/1
6
local
0.0.0.0/0 NAT Gateway
Destination Target
172.31.0.0/1
6
local
0.0.0.0/0 Internet
Gateway
Route Tables
Attach Internet Gateway to Amazon VPC
Setup a Public Subnet with
• Route to Internet Gateway
• NAT Gateway
Setup Private Subnet with
• Fargate Task
• Route to NAT Gateway
Security Group to allow outbound traffic
Type Port Destination
All Traffic ALL 0.0.0.0/0
Outbound Security Group Rules

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Architecture Reference: Internet facing ELB w/ VPC
Public subnet Private subnet
Fargate
TaskENI
Private IP
172.31.1.164
:8080
ALB
Public IP
208.57.73.13
:80
172.31.0.0/16
172.31.2.0/24 172.31.1.0/24
Task in private subnet with private IP
ALB in public subnet with public IP
Make sure the AZs of the two subnets
match
ALB security group to allow inbound traffic
from internet
Task security group to allow inbound traffic
from the ALB’s security group
Task Security GroupALB Security Group
Type Port Source
HTTP 80 0.0.0.0/0
Inbound Rule
Type Port Source
Custom TCP 808
0
ALB Security Group
Inbound Rule
us-east-1a us-east-1a

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Sample Application – Task Definition
Amazon ECS Task Definition: Visibility
{
"family": "scorekeep",
"cpu": "1 vCpu",
"memory": "2 gb",
"networkMode":"awsvpc",
"taskRoleArn": "arn:aws:…",
"executionRoleArn”: “arn:…”
"requiresCompatibilities":
[
"FARGATE"
],
"containerDefinitions": [
{
"name": "scorekeep-frontend",
"image":“xxx.dkr.ecr…frontend",
"cpu": 256,
"memoryReservation": 512,
"portMappings" : [
{ "containerPort": 8080 }
],
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "scorekeep",
"awslogs-region": "us-east-1",
"awslogs-stream-prefix":
"scorekeep/frontend”
}
}
}
]}
{
"name": "scorekeep-api",
"image":“xxx.dkr.ecr…api",
"cpu": 768,
"memoryReservation": 512,
"portMappings" : [
{ "containerPort": 5000 }
],
"environment": […], #env var
"logConfiguration": {
"logDriver": "awslogs",
"options": {
“awslogs-group": "scorekeep",
"awslogs-region": "us-east-1",
"awslogs-stream-prefix":
"scorekeep/api”
}
}
},

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Frontend Server
Container
Angular + Nginx
API Server
Container
Java
Port
8080
Port
5000
Internet
Sample Application - Architecture

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
AWS Services for Containerized Microservices
Storage & Database
Amazon
DynamoDB
Amazon ElastiCache
Amazon Relational
Database Service
(Amazon RDS)
Amazon
Simple Storage
Service
(Amazon S3)
Logging & Monitoring
Amazon
CloudWatch
AWS
CloudTrail
AWS X-Ray
Application Integration
Amazon
MQ
Amazon
Simple
Notification
Service
(Amazon
SNS)
Amazon
Simple Queue
Service
(Amazon
SQS)
AWS Step
Functions
Developer Tools
AWS
CodeBuild
AWS
CodePipeline
AWS
Cloud9
Networking & API Proxy
Amazon API
Gateway
Elastic Load
Balancing
Amazon
Route 53
Compute
Amazon
EKS
Amazon
ECS
Amazon
ECR
Fargate
AWS Elastic
Beanstalk

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Investor
protection
Market
integrity
FINRA MISSION

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
brokers
12 634,000
markets/
exchanges
firms
3,800
VOLUME

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
FINRA maintains
150+
applications running in AWS
FINRA processes up to
135 billion
market events per day
FINRA processes and analyzes
trillions
of nodes and edges
FINRA manages approximately
30 Petabytes
of storage
FINRA runs up to
50,000 compute nodes
per day
BIG DATA

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Agility
ON-PREMISE AWS EC2-ELB
Infrastructure costs
Monthly release
cycles
Dependency
management
Scalability
Parity
Many microservicesHigh costs
Slow build time Slow deployment
time (puppet)
Parity
PROBLEMS

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Easily portable
Faster
Simple and lightweight
Cost efficient
Productivity
FIVE MAIN BENEFITS
CONTAINERS TO THE RESCUE

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
CONTAINER MANAGEMENT

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Simple and Easy to use
Integration with AWS
Services
Low learning curve
No fine grained container
placement strategies
Cluster scaling overheads
Periodic AMI update
overheads
PROS CONS USE CASES
Standalone Tasks
Batch Jobs
Web Applications – UI, API,
Microservices
ECS

39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Serverless
No Cluster Scaling overheads
No AMI updates
No Encrypted Storage
No EFS integration
ENI per task, capped by
account level ENI limits
PROS CONS USE CASES
FARGATE
Web Applications – UI, API,
Microservices
Standalone tasks and Batch
Jobs
Stateless Applications (or
managing states externally
using s3/RDS/DynamoDB)

40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Provides more flexibility
Large Open Source
Ecosystem
Higher learning curve
More engineering effort
Cluster Management
Complexity
PROS CONS USE CASES
Hybrid Applications
Big Data processing
Platform Applications
EKS

41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Simple and easy to use
Good fit for majority of FINRA use cases [Web Application, MicroServices, Standalone tasks and Batch Jobs]
Easy migration from EC2-ELB to ECS-ALB
Cluster per business domain
SOLUTION CHOSEN

42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Many apps
Automation
Desired architecture
Security and compliance
Best practices
Rollout
SIX MIGRATION CHALLENGES

43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
DESIRED ARCHITECTURE
/app1 /myapp.finra.org/app3/app2
path-based routing host-based routing
Springboot Microservice
Service
Task
Identity Gateway Springboot Splunk Sidecar
Angular App
Service
Task
Identity Gateway NodeJS Splunk Sidecar
ECS
EC2 instance contents EC2 instance contents EC2 instance contents
Daemon Collectd daemon Host metrics Daemon Collectd daemon Host metrics Daemon Collectd daemon Host metrics
Monolithic App
Service
Task
Identity Gateway Tomcat Splunk Sidecar
443/8443 listener
Application Load Balancing (ALB)
Amazon EC2 Container Registry (ECR)
pull images
Availability Zone #1 Availability Zone #2

44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
FINRA ECS Optimized AMI on ECS Cluster
CIS Security Specification for Docker
Security scanning of Containers
(Cloud Passage Halo agent)
AMI
Amazon ECS Optimized AMI
FINRA ECS Optimized AMI

45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Amazon Linux
FINRA Base Linux
FINRA IMAGES
Reusable, compliant FINRA Base Images
App images focus on packaging code and
configuration
My Tomcat Apps
MY APPLICATION IMAGES
My Springboot Apps My Angular Apps
DOCKER IMAGES
Java
Nodejs
Apache
Splunk
Tomcat
Identity Gateway

46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Provision
AWS Tools and Libraries
CloudFormation ECS CLI AWS SDK
Best Practices FINRA Tools and Libraries
DNS APIsSecrets ManagementStandards Compliance Security Defaults
ECS Stack
Amazon Elastic
Container
Service
Auto Scaling
ALB Stack
Load Balancer Secure Listeners
Default Target
Groups
ECR Repos Service Stack
Task Definition Routing Rules
Service Target Groups
INFRASTRUCTURE AUTOMATION

47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
ECS Cluster Pipeline
Source Repository Build and Archive
Infrastructure code
S3
Dev
Provision ECS Cluster
QA
Provision ECS Cluster
Prod
Provision ECS Cluster
ALB Pipeline
Source Repository Build and Archive
Infrastructure code
S3
Dev
Provision ECS Cluster
QA
Provision ECS Cluster
Prod
Provision ECS Cluster
App/Service Pipeline
Source Repository Build App
Code
S3
Dev
Provision
Artifactory
Build App
Docker Image
ECR
ECR
Service Stack
QA
ProvisionECR
Service Stack
Prod
ECR
Service Stack
Build & Tag Docker Containers Once
Push & Deploy Containers to any account multiple times
DELIVERY PIPELINE

48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
100+
clusters
200+
EC2s
500+
active tasks
300+
services
PRODUCTION METRICS

49. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Blue Green Deployment
• ALB DNS switch
• Target Group switch
Zero Downtime Deployment
• Rolling Update
ECS DEPLOYMENT STRATEGIES

50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Right size Cluster and Services
Spread tasks
• Availability zones - Placement Strategy
• Instances - Placement Constraints
Logging and monitoring
BEST PRACTICES

51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Serverless Adoption with Fargate
EKS Adoption for Big Data Processing
Open Source Provision tool (Q4 2019)
FUTURE

52. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
http://technology.finra.org/opensource.html
Temporary access to
EC2/RDS
Gatekeeper
Secrets management in
AWS
Fidelius
Create resources in AWS
Provision
Temporary token for AWS
CloudPass
Security group manager
Portus
GO API to Upload RPM to
Yum
yum-nginx-api
Monitor AWS service limits
Aphelion
AVAILABLE
COMING SOON
FINRA OPEN SOURCE

53. Thank you!
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Harsha Nippani
nippanih@amazon.com
Latha Nagaraj
Latha.NagarajSai@finra.org