At AWS re:Invent, we have launched support for blue/green deployments for services hosted using AWS Fargate and Amazon Elastic Container Service (Amazon ECS). Blue/green deployments help you minimize downtime during application updates. They allow you to launch a new version of your application alongside the old version and test the new version before you reroute traffic to it. You can also monitor the deployment process and, if there is an issue, quickly roll back. In this workshop, you will create a new service in AWS Fargate that uses AWS CodeDeploy to manage the deployments, testing, and traffic cutover for you.

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Kwunhok Chan, Solutions Architect
khchan@amazon.com
Build a CI/CD Pipeline for
your Containers

2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Agenda
• Containers, Amazon ECS and AWS Fargate
• CI/CD for modern applications
• Continuous Integration
• Continuous Deployment

3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
At first there was
Amazon EC2

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Then Docker!
Customers started containerizing applications
within EC2 instances
EC2 Instance
Containers

5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Containers made it easy to build and scale
cloud-native applications

6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Customers needed an easier way to manage large clusters of
instances and containers

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AMAZON ELASTIC CONTAINER SERVICE
Cluster Management as a hosted service

8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ECS
Agent
Docker
Agent
OS
EC2 Instance
But cluster management is only half the equation…

9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Managing instance fleets is hard work too!
Patching and Upgrading OS, agents, etc.
Scaling the instance fleet for optimal utilization

10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ECS
Agent
Docker
Agent
OS
EC2 Instance
ECS
Agent
Docker
Agent
OS
EC2 Instance
ECS
Agent
Docker
Agent
OS
EC2 Instance
Customers wanted to run containers without having to manage EC2 instances
Scheduling and Orchestration
Cluster Manager Placement Engine

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Your Docker
Containers
NO INSTANCES TO MANAGE
No EC2 Instances to provision, scale or manage
ELASTIC
Scale up & down seamlessly. Pay only for what you use
INTEGRATED
with the AWS ecosystem: VPC Networking,
Elastic Load Balancing, IAM Permissions, Cloudwatch and more.
AWS FARGATE

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Container Services Landscape
MANAGEMENT
Deployment, Scheduling,
Scaling & Management
HOSTING
Where the containers run
Amazon EC2
IMAGE REGISTRY
Container Image Repository

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Hands-on http://bit.ly/2RfNqjx

14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CI/CD for Modern Applications

15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Release process stages
Source Build Test Production

16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Release process stages
Source Build Test Production
• Check-in source
code such as
.java files
• Peer review new
code

17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Release process stages
Source Build Test Production
• Check-in source
code such as
.java files
• Peer review new
code
• Compile code
• Unit tests
• Style checkers
• Create container
images and
function
deployment
packages

18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Release process stages
Source Build Test Production
• Integration tests
with other
systems
• Load testing
• UI tests
• Security testing
• Check-in source
code such as
.java files
• Peer review new
code
• Compile code
• Unit tests
• Style checkers
• Create container
images and
function
deployment
packages

19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Release process stages
Source Build Test Production
• Integration tests
with other
systems
• Load testing
• UI tests
• Security testing
• Check-in source
code such as
.java files
• Peer review new
code
• Compile code
• Unit tests
• Style checkers
• Create container
images and
function
deployment
packages
• Deployment to
production
environments
• Monitor code in
production to
quickly detect
errors

20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Release process stages
Source Build Test Production
• Integration tests
with other
systems
• Load testing
• UI tests
• Security testing
• Check-in source
code such as
.java files
• Peer review new
code
• Compile code
• Unit tests
• Style checkers
• Create container
images and
function
deployment
packages
• Deployment to
production
environments
• Monitor code in
production to
quickly detect
errors

21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Release process stages
Source Build Test Production

22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Release process stages
Source Build Test Production

23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Effects of CI/CD
Source: 2018 DORA State of DevOps Report

24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Effects of CI/CD
Source: 2018 DORA State of DevOps Report
Deployment frequency Weekly – monthly Hourly – daily

25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Effects of CI/CD
Source: 2018 DORA State of DevOps Report
Deployment frequency Weekly – monthly Hourly – daily
Change lead time One – six months One – seven days

26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Effects of CI/CD
Source: 2018 DORA State of DevOps Report
Deployment frequency Weekly – monthly Hourly – daily
Change lead time One – six months One – seven days
Change failure rate 46 - 60% 0 - 15%

27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Effects of CI/CD
Source: 2018 DORA State of DevOps Report
Deployment frequency Weekly – monthly Hourly – daily
Change lead time One – six months One – seven days
Change failure rate 46 - 60% 0 - 15%
48% of
software
teams

28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Continuous integration goals
1. Automatically kick off a new release when new code is checked in
2. Build and test code in a consistent, repeatable environment
3. Continually have an artifact ready for deployment
4. Continually close feedback loop when build fails

29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CodePipeline
• Continuous delivery service for fast and reliable
application updates
• Model and visualize your software release
process
• Builds, tests, and deploys your code every time
there is a code change
• Integrates with third-party tools and AWS

30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CodePipeline: Supported sources
Pick object or folder
Amazon Simple
Storage Service
(Amazon S3)
Automatically kick off release and pull latest source code
Pick branch
AWS CodeCommit
GitHub

31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CodePipeline: Supported sources
AWS CodePipeline now uses
Amazon Elastic Container Registry
(Amazon ECR) as a pipeline source
New

32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CodePipeline: Supported sources
Pick object or folder
Amazon S3
Pick Docker tag
Amazon ECR
Automatically kick off release and pull latest source code
Pick branch
AWS CodeCommit
GitHub

33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CodePipeline: ECR source action
Source code:
“master” branch
ECR repository:
“release” tag
Build
stage
Deploy
stages

34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CodePipeline: Supported triggers
Automatically kick off release
Amazon CloudWatch Events
• Scheduled (nightly release)
• AWS Health events (Fargate
platform retirement)
Available in CloudWatch Events
console, API, SDK, CLI, and
AWS CloudFormation
Webhooks
• DockerHub
• Quay
• Artifactory
Available in CodePipeline API,
SDK, CLI, and CloudFormation

35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CodeBuild
• Fully managed build service that compiles
source code, runs tests, and produces
software packages
• Scales continuously and processes multiple
builds concurrently
• No build servers to manage
• Pay by the minute, only for the compute
resources you use
• Monitor builds through CloudWatch Events

36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CodeBuild
• Each build runs in a new Docker container
for a consistent, immutable environment
• Docker and AWS CLI are installed in every
official CodeBuild image
• Provide custom build environments suited to
your needs through the use of Docker
images

37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CodeBuild: Docker buildspec
version: 0.2
phases:
build:
commands:
- $(aws ecr get-login --no-include-email)
- docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG .
- docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $ECR_REPO:$IMAGE_TAG
- docker push $ECR_REPO:$IMAGE_TAG

38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Continuous integration goals
1. Automatically kick off a new release when new code is checked in
2. Build and test code in a consistent, repeatable environment
3. Continually have an artifact ready for deployment
4. Continually close feedback loop when build fails

39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Continuous deployment goals
Source Build Test Production

40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Continuous deployment goals
1. Automatically deploy new changes to staging environments for testing
2. Deploy to production safely without impacting customers
3. Deliver to customers faster: Increase deployment frequency, and
reduce change lead time and change failure rate

41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CodeDeploy
• Automates code deployments to any instance
and Lambda
• Handles the complexity of updating your
applications
• Avoid downtime during application deployment
• Roll back automatically if failure detected
• Deploy to Amazon EC2, Lambda, or on-
premises servers

42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CodeDeploy
AWS CodeDeploy now automates blue-
green deployments to AWS Fargate and
Amazon Elastic Container Service (ECS)
New

43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CodeDeploy-ECS blue-green deployments
• Provisions “green” tasks, then flips traffic at the load balancer
• Validation “hooks” enable testing at each stage of the
deployment
• Fast rollback to “blue” tasks in seconds if case of hook failure
or CloudWatch alarms
• Monitor deployment status and history via console, API,
Amazon SNS notifications, and CloudWatch Events
• Use “CodeDeploy-ECS” deploy action in CodePipeline or “aws
ecs deploy” command in Jenkins

44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CodeDeploy-ECS appspec
version: 1.0
Resources:
- TargetService:
Type: AWS::ECS::Service
Properties:
- TaskDefinition: "my_task_definition:8"
LoadBalancerInfos:
- ContainerName: "SampleApp"
ContainerPort: 80
Hooks:
- BeforeInstall: "LambdaFunctionToExecuteAnythingBeforeNewRevisionInstalltion"
- AfterInstall: "LambdaFunctionToExecuteAnythingAfterNewRevisionInstallation"
- AfterAllowTestTraffic: "LambdaFunctionToValidateAfterTestTrafficShift"
- BeforeAllowTraffic: "LambdaFunctionToValidateBeforeTrafficShift"
- AfterAllowTraffic: "LambdaFunctionToValidateAfterTrafficShift"

45. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CodeDeploy-ECS blue-green deployment
Application
Load
Balancer
Production
traffic
listener (port
80)
Target group
1
Blue tasks:
v1 code
100%
Prod
traffic
Fargate
service

46. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CodeDeploy-ECS blue-green deployment
Application
Load
Balancer
Test traffic
listener (port
9000)
Production
traffic
listener (port
80)
Target group
2
Target group
1
Blue tasks:
v1 code
100%
Prod
traffic
Fargate
service

47. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CodeDeploy-ECS blue-green deployment
Application
Load
Balancer
Test traffic
listener (port
9000)
Production
traffic
listener (port
80)
Target group
2
Target group
1
Blue tasks:
v1 code
Green tasks:
v2 code
100%
Prod
traffic
Provision green tasks
Fargate
service

48. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CodeDeploy-ECS blue-green deployment
Application
Load
Balancer
Test traffic
listener (port
9000)
Production
traffic
listener (port
80)
Target group
2
Target group
1
Blue tasks:
v1 code
Green tasks:
v2 code
100%
Test
traffic
100%
Prod
traffic
Run hook against test endpoint before green tasks receive prod traffic
Fargate
service

49. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CodeDeploy-ECS blue-green deployment
Application
Load
Balancer
Test traffic
listener (port
9000)
Production
traffic
listener (port
80)
Target group
2
Target group
1
Blue tasks:
v1 code
Green tasks:
v2 code
100%
Prod
traffic
Flip traffic to green tasks, rollback in case of alarm
0% Prod
traffic
Fargate
service

50. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CodeDeploy-ECS blue-green deployment
Application
Load
Balancer
Test traffic
listener (port
9000)
Production
traffic
listener (port
80)
Target group
2
Target group
1
Green tasks:
v2 code
100%
Prod
traffic
Drain blue tasks
Fargate
service

51. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Container image tagging for deployments
• Docker tags are resolved when each container starts, not
just during deployments
• Deploying “latest” or “prod” can result in untested code in
production after a scale-out event
• Use unique “immutable” tags for deployments

52. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Container image tagging for deployments
Image: “latest” tag
Fargate
service
ECR
repository
Image: sha256@11111... (“latest”)

53. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Container image tagging for deployments
Image: “latest” tag
Fargate
service
Build pushes new “latest” image
ECR
repository
Image: sha256@11111...
Image: sha256@22222... (“latest”)

54. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Container image tagging for deployments
Image: “latest” tag
Fargate
service
Service scales up, launching new tasks
ECR
repository
Image: sha256@11111...
Image: sha256@22222... (“latest”)

55. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Container image tagging for deployments
Deploy using immutable tags
{
"name": "sample-app",
"image": "amazon/amazon-ecs-
sample@sha256:3e39d933b1d948c92309bb583b5a1f3d28f0119e1551ca1fe538ba414a41af48d"
}
{
"name": "sample-app",
"image": "amazon/amazon-ecs-sample:build-b2085490-359f-4eaf-8970-6d1e26c354f0"
}
SHA256 Digest
Build ID

56. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Container image tagging for deployments
Compute immutable tags during build
SHA256 Digest
export IMAGE_URI=`docker inspect --format='{{index .RepoDigests 0}}'
my_image:$IMAGE_TAG
Example Result:
amazon/amazon-ecs-sample@sha256:3e39d933b...
Build ID
export IMAGE_TAG=build-`echo $CODEBUILD_BUILD_ID | awk –F":" ‘{print $2}'`
Example Result:
build-b2085490-359f-4eaf-8970-6d1e26c354f0

57. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Container image tagging for deployments
Image: “build-11111” tag
Fargate
service
ECR
repository
Image: sha256@11111... (“build-11111”)

58. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Container image tagging for deployments
Image: “build-11111” tag
Fargate
service
Build pushes new image tagged with new build ID
ECR
repository
Image: sha256@11111... (“build-11111”)
Image: sha256@22222... (“build-22222”)

59. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Container image tagging for deployments
Image: “build-11111” tag
Fargate
service
Service scales up, launching new tasks
ECR
repository
Image: sha256@11111... (“build-11111”)
Image: sha256@22222... (“build-22222”)

60. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Container image tagging for deployments
Image: “build-22222” tag
Fargate
service
Deployment updates service’s task definition, replacing tasks
ECR
repository
Image: sha256@11111... (“build-11111”)
Image: sha256@22222... (“build-22222”)

61. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Continuous deployment goals
1. Automatically deploy new changes to staging environments for testing
2. Deploy to production safely without impacting customers
3. Deliver to customers faster: Increase deployment frequency, and
reduce change lead time and change failure rate

62. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Hands-on http://bit.ly/2RfNqjx

63. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!
khchan@amazon.com