CI/CD for Containers: A Way Forward for Your DevOps Pipeline

P U B L I C S E C T O R
S U M M I T
WASHINGTON, DC

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
CI/CD for Serverless and
Containerized Applications
Len Henry
Senior Solutions
Architect
AWS/WWPS
Education
3 0 1 5 9 4
Justin Almquist
Senior Software
Engineer &
Development Team
Lead
Computing and
Analytics
Troy Zuroske
Software
Engineer/Cloud
Architect Computing
and Analytics

S U M M I T
Agenda
CI/CD for modern applications
Continuous integration
Continuous deployment
Infrastructure as code
CI/CD for a Real-Time Deep Learning Pipeline

S U M M I T
What is a modern application?

S U M M I T
Approaches to modern application development
• Simplify environment management
• Reduce the impact of code changes
• Automate operations
• Accelerate the delivery of services
• Gain insight across resources and applications
• Protect customers and the business

S U M M I T
• Simplify environment management with serverless technologies
• Reduce the impact of code changes with microservice architectures
• Automate operations by modeling applications & infrastructure as
code
• Accelerate the delivery of services with CI/CD
• Gain insight across resources and applications by enabling
observability
• Protect customers and the business with end-to-end security &
compliance

S U M M I T
Serverless containers
Long-running
Abstracts the OS
Fully managed orchestration
Fully managed cluster scaling
Serverless functions
Event-driven
Many language runtimes
Data source integrations
No server management

S U M M I T
Release process stages
Source Build Test Production

S U M M I T
Release process stages
• Integration tests
with other
systems
• Load testing
• UI tests
• Security testing
• Check-in source
code such as .java
files
• Peer review new
code
• Compile code
• Unit tests
• Style checkers
• Create container
images and
function
deployment
packages
• Deployment to
production
environments
• Monitor code in
production to
quickly detect
errors

S U M M I T
Effects of CI/CD
Source: 2018 DORA State of DevOps Report
Deployment frequency Weekly – monthly Hourly – daily
Change lead time One – six months One – seven days
Change failure rate 46-60% 0-15%

S U M M I T
Pillars of releasing modern applications

S U M M I T
Continuous
integration

S U M M I T
Continuous integration goals

S U M M I T
Continuous integration goals
1. Automatically kick off a new release when new code is checked in
2. Build and test code in a consistent, repeatable environment
3. Continually have an artifact ready for deployment
4. Continually close feedback loop when build fails

S U M M I T
AWS CodePipeline
• Continuous delivery service for fast and reliable
application updates
• Model and visualize your software release
process
• Builds, tests, and deploys your code every time
there is a code change
• Integrates with third-party tools and AWS

S U M M I T
AWS CodePipeline: Supported sources
Pick branch
AWS CodeCommit
GitHub
Pick object or folder
Amazon Simple
Storage Service
(Amazon S3)
Pick Docker tag
Amazon Elastic
Container
Registry (Amazon
ECR)
Automatically kick off release and pull latest source code

S U M M I T
AWS CodeBuild
• Fully managed build service that compiles
source code, runs tests, and produces software
packages
• Scales continuously and processes multiple
builds concurrently
• No build servers to manage
• Pay by the minute, only for the compute
resources you use
• Monitor builds through Amazon CloudWatch
Events

S U M M I T
AWS CodeBuild
• Each build runs in a new Docker container for a
consistent, immutable environment
• Docker and AWS Command Line Interface
(AWS CLI) are installed in every official
CodeBuild image
• Provide custom build environments suited to
your needs through the use of Docker images

S U M M I T
Continuous
deployment
Continuous
integration

S U M M I T
Continuous deployment goals

S U M M I T
Continuous deployment goals
1. Automatically deploy new changes to staging environments for
testing
2. Deploy to production safely without impacting customers
3. Deliver to customers faster: Increase deployment frequency and
reduce change lead time and change failure rate

S U M M I T
AWS CodeDeploy
• Automates code deployments to any instance
and AWS Lambda
• Handles the complexity of updating your
applications
• Avoids downtime during application
deployment
• Rolls back automatically if failure detected
• Deploys to Amazon Elastic Compute Cloud
(Amazon EC2), Lambda, or on-premises servers

S U M M I T
CodeDeploy-Lambda deployments
• Shifts traffic using Lambda function weighted aliases
• Choose canary (“shift 10% of traffic for 10 minutes, then shift
rest”) or linear (“shift 10% more traffic every 10 minutes”)
• Validation “hooks” enable testing at each stage of the
deployment
• Fast rollback in seconds if case of hook failure or CloudWatch
alarms
• Monitor deployment status and history via console, API, Amazon
Simple Notification Service (Amazon SNS) notifications, and
CloudWatch Events

S U M M I T
AWS CodeDeploy
AWS CodeDeploy now automates
blue-green deployments to AWS
Fargate and Amazon Elastic
Container Service (Amazon ECS)

S U M M I T
Container image tagging for deployments
• Docker tags are resolved when each container starts, not just
during deployments
• Deploying “latest” or “prod” can result in untested code in
production after a scale-out event
• Use unique “immutable” tags for deployments

S U M M I T
Infrastructure
as code
Continuous
deployment
Continuous
integration

S U M M I T
Infrastructure as code goals
Source Build Test
Productio
n

S U M M I T
Infrastructure as code goals
1. Make infrastructure changes repeatable and predictable
2. Release infrastructure changes using the same tools as code
changes
3. Replicate production environment in a staging environment to
enable continuous testing

S U M M I T
Continuous testing with infrastructure as code
Validate an artifact
(Build stage)
• Unit tests
• Static analysis
• Mocked dependencies and
environments
• Vulnerability image scans
Validate an environment
(Test stages)
• Integration tests against real
dependencies and real
environments
• Load testing
• Penetration testing
• Monitoring to test impact of
deployments on
environment

S U M M I T
Release infrastructure-as-code
“Master”
branch
Prepare
template
Create & execute
change set
Create & execute
change set

S U M M I T
Model function environments with AWS
Serverless Application Model (SAM)
https://aws.amazon.com/serverless/sam/
• Open source framework for building serverless
applications on AWS
• Shorthand syntax to express functions, APIs,
databases, and event source mappings
• Transforms and expands SAM syntax into AWS
CloudFormation syntax on deployment
• Supports all AWS CloudFormation resource types

S U M M I T
Model container environments with AWS
Cloud Development Kit (CDK)
• Open source framework to define cloud
infrastructure in Typescript
• Provides library of higher-level resource types
(“construct” classes) that have AWS best
practices built in by default, packaged as npm
modules
• Provisions resources with AWS CloudFormation
• Supports all AWS CloudFormation resource
types
AWS
CDK
https://awslabs.github.io/aws-cdk

S U M M I T
CDK template
import ec2 = require('@aws-cdk/aws-ec2');
import ecs = require('@aws-cdk/aws-ecs');
import cdk = require('@aws-cdk/cdk');
class BonjourFargate extends cdk.Stack {
constructor(parent: cdk.App, name: string, props?: cdk.StackProps) {
super(parent, name, props);
const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 });
const cluster = new ecs.Cluster(this, 'Cluster', { vpc });
new ecs.LoadBalancedFargateService(
this, "FargateService", {
cluster,
image: ecs.DockerHub.image("amazon/amazon-ecs-sample"),
});
}
}
const app = new cdk.App();
new BonjourFargate(app, 'Bonjour');
app.run();

S U M M I T
Model pipelines with AWS CDK
• Minimize copy-and-paste by using object-oriented language
• Define microservice pipeline “shape” in one class, then re-use it
across many pipelines
• AWS CDK includes many high-level constructs for modeling a
CodePipeline pipeline, including automatically configuring AWS
Identity and Access Management (IAM) role policies

S U M M I T
Draft
CI/CD for a Real-Time Deep Learning Pipeline
Justin Almquist,
Senior Software Engineer PNNL
Troy Zuroske,
Software Engineer PNNL
June 19, 2019

S U M M I T
41
Introductions
Justin Almquist
Senior Software Engineer &
Development Team Lead
Computing and Analytics
Troy Zuroske
Software Engineer/Cloud Architect
Computing and Analytics
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

S U M M I T
42
PNNL at a Glance
4,500
Employees
$987M
Budget
1,127
Publications
204
Inventions
64
Patents

S U M M I T
43
Agenda
• Streaming deep learning image analysis pipeline
• Deploying models with Docker
• Reproducible deploys of GPU-enabled deep learning models
• Infrastructure as Code
• Automation for deploying complex serverless architecture
• Multi-account CI/CD
• Quick deploys and tight feedback for rapid development
• Questions?

S U M M I T
44
Streaming Deep Learning Image Analysis Pipeline
• Goal: Enable users to find relevant imagery
• Models: Junk filtering, avoid unnecessary
labeling, easily add new models
• Scale: 100+ images/sec
• Near real-time: <60 seconds end-to-end
NSFW Crowds
ImageNet
Indoor/Outdoor
Geolocation
NSFW?
Indoor?
Image Labels

S U M M I T
45
Reactive, Message-Driven Architecture
How?
 Amazon Simple Queue Service
(Amazon SQS) and AWS Lambda
 Amazon DynamoDB for state
 Pipeline Lambda for orchestration logic
Why?
 Robust – stands up to bursts of data
without falling over
 Reliable – tolerates failures
 Scalable – scale up and down based
on streaming input
 Maintainable – easy to evolve
individual components
Images
Model
Pipeline Lambda
Model Model Model
Amazon DynamoDB

S U M M I T
46
Deep Learning Models with Docker
Primary goal: Reproducible deploys on GPUs
 Today, tomorrow, next year
Docker GPU stack
 Application code and model
 Tensorflow-GPU and other Python packages
 NVIDIA Docker container
 NVIDIA Docker
 AWS Deep Learning AMI
Capture version dependencies
 Python packages
 System packages
 AMI
 The CUDA driver on the host must newer or equal to Docker container!

S U M M I T
47
Deep Learning Models - Deployment
AWS Lambda?
 Some models and libraries don’t fit in
available storage space
 GPUs are faster and more cost-
effective (efficient)
Amazon SageMaker?
 Supports REST API, but we needed
Amazon SQS workers
AWS Fargate?
 Ideal solution, but no GPU support
Amazon ECS?
 Complex auto-scaling: Two levels –
EC2 cluster and ECS service, Lambda
helper
 Slow auto-scaling: 20 minutes from
command to running worker
Amazon EC2 ✅
 Full control of instance/GPU type
 Simpler, faster auto-scaling than ECS
 Autoscaling based on queue length
(scale out) and worker utilization
(scale in)

S U M M I T
48
Infrastructure as Code (IaC)
• Capture AWS resources as code
 Machine readable, under revision control
• Required with Reactive, Serverless
 Our architecture has over 100 AWS resources
– manual is not an option
 Reproducibility, fewer errors, less time
 Enables continuous deployments (CD)
• AWS CloudFormation
 Alternative: Terraform
 Future: AWS CDK – program infrastructure in a real programming language
• IAM roles
• VPC and subnets
• Lambda functions
• Amazon SNS topics
• Amazon SQS
queues
• Amazon S3 buckets
• DynamoDB tables
• CloudWatch
Dashboards
• CloudWatch log
Groups
• CloudWatch alarms
• Autoscaling groups
• Autoscaling policies

S U M M I T
49
AWS CloudFormation Lessons Learned
Capture everything required to deploy application from scratch
 Use conditions if necessary
Give every developer their own stack
 Use the username in the identifiers for uniqueness
Use the linter to validate templates – cfn-lint
Use YAML instead of JSON
Use AWS SAM for serverless resources
Use nested stacks to reduce copy/paste
 Used extensively when working with Deep Learning models
Script the deploy
 Shell script + environment-specific config files
 $ ./deploy.sh development.cfg

S U M M I T
50
Development Process
• Git feature branch workflow
• Pull requests for code review
• Automated tests for PR feedback
• Merges result in automated deploys to our test/staging environment
Developer
Creates Feature
Branch
Developer
Commits
Changes
Developer Creates
/ Updates Pull
Request
Code Reviewers
Inspect Changes
Developer
Merges Pull
Request
CI System
Builds and
Tests Software
CI System
Deploys to
Staging
CI System
Builds and
Tests Software
Developer
Pushes
Changes
PassPassFail
Feedback Approval

S U M M I T
51
Dev
Alice
Production
Bob
Production
Registry
Registry
Alice Bob
Tools On Premises
Build
Deploy
Commit
Test/Staging
Test
Registry
Multi-Developer and Multi-Account CI/CD Pipeline

S U M M I T
52
CI/CD
Model Repo Lambda Repo
Model Build Lambda Build
Deploy
• Repos
 Model repo stores models and service wrappers
 Lambda repo has lambda code and AWS CloudFormation
• Builds
 Model build produces Docker images for each of the
models, and an inventory of all the image tags
 Lambda build runs linter, unit tests and produces a
software package for deploys
 Model build triggers lambda build
 Lambda build triggers deploy
• Deploy
 Create/update AWS CloudFormation stack
 Pull/push and retag images
 Deploy new images
Model Image Lambda Package

S U M M I T
53
CI/CD for Docker-ized Models
Models and wrapper code are packaged as Docker images
Images stored in Amazon ECR
Image tagging
 Images tagged with build number during builds
 Images retagged with environment name during deploys
Cross account Amazon ECR permissions to move images between accounts
Amazon ECR policies to delete old/unused images
 Keep images from last N builds
 Keep images tagged for an environment: ‘development,’ ‘test,’ ‘production’
 Delete untagged images

S U M M I T
54
Conclusions
• Capture all dependencies as code
 requirements.txt
 Dockerfiles
 AWS CloudFormation templates
 Scripts
 Config files
• Automate everything
 Build, test, deploy
 Reduce mistakes, iterate faster, improve reproducibility
• Previous two steps are building blocks for CI/CD
 Reproducible builds/deploys
 Easily deploy into different stacks/environments (developer, test, stage, prod, etc.)

S U M M I T
55
Thank you!
Cindy Hewitt
Len Henry
• Gideon Juve
• Bill Vilwock
• Jordan Hendry
• Mike Giardinelli
• Ralph Perko

Thank you!
S U M M I T

S U M M I T

CI/CD for Containers: A Way Forward for Your DevOps Pipeline

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a CI/CD for Containers: A Way Forward for Your DevOps Pipeline

Similar a CI/CD for Containers: A Way Forward for Your DevOps Pipeline (20)

Más de Amazon Web Services

Más de Amazon Web Services (20)

CI/CD for Containers: A Way Forward for Your DevOps Pipeline