The document discusses best practices for building application development release pipelines, including: - Implementing continuous integration/continuous delivery (CI/CD) practices like committing code frequently and building on every commit. - Deploying code to running environments for further testing before production. - Storing all code, including applications, infrastructure, and documentation in code repositories. - Starting with continuous delivery and gated deployments, then moving to continuous deployment with excellent testing. - Conducting code reviews to ensure code quality and understandability.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Travis Williams, Enterprise Solutions Architect
DevOps on AWS:
Accelerating Software Delivery with the
AWS Developer Tools
November 30, 2016

2. https://secure.flickr.com/photos/mgifford/4525333972
Why are we
here today?

3. Software moves faster today
Software creation and distribution is
easier and faster than ever:
• Startups can now take on giants with little to
no funding ahead of time
• Getting your software into the hands of
millions is a download away
• Your ability to move fast is paramount to your
ability to fight off disruption

4. Old software delivery model
The software delivery model has drastically changed
New software delivery model

5. What tools do you need to move fast?
Releasing software in this new software driven world
requires a number of tools:
• Tools to manage the flow of your software development
release process
• Tools to properly test and inspect your code for defects
and potential issues
• Tools to deploy your applications

6. First, we need to
understand a little bit
about software release
processes
https://www.flickr.com/photos/jurvetson/5201796697/

7. • Integration
tests with
other systems
• Load testing
• UI tests
• Penetration
testing
Release processes have four major phases
Source Build Test Production
• Check-in
source code
such as .java
files.
• Peer review
new code
• Compile code
• Unit tests
• Style checkers
• Code metrics
• Create
container
images
• Deployment
to production
environments

8. Release processes levels
Source Build Test Production
Continuous integration
Continuous delivery
Continuous deployment

9. We move pretty fast at Amazon:
In 2014:
• Thousands of service teams across Amazon
• + Building microservices
• + Practicing continuous delivery
• + Many environments (staging, beta, production,
multiple regions)
=50 million deploys

10. microservices + 2 pizza teams

11. DevOps: Culture + Practices + Tools
Each 2-pizza team “owns” their
product:
• Creates product (software
typically)
• Handles Q/A of that product
• Responds to issues, is on-call
• “you build it, you run it”
• Supports service & tracks/goals
against business and technical
metrics
https://secure.flickr.com/photos/lox/9408028555

12. DevOps: Culture + Practices + Tools
Each 2-pizza team’s practices
largely open so far as standards are
met:
• Agile? Scrum? Daily standups?
Weekly? None? Whatever you
works for your team!
• No centralized change
management board/team/
approval, but tools that require a
degree of sign-off/process review
https://secure.flickr.com/photos/lox/9408028555

13. DevOps: Culture + Practices + Tools
Each 2-pizza team developers given
a “box of tools”:
• Use these, or operationalize your
own:
• Time spent on operations is less
time to spend on development
• Less time spent on development
is increased risk of missing goals
• Tools provide “guard rails” that
enforce best/better practices
• Tools maintained by other 2-pizza
teams
https://secure.flickr.com/photos/lox/9408028555

14. 2 Pizza Team Responsibility Venn Diagram
Responsible for
THEIR
PRODUCT
Deployment tools
CI/CD tools
Monitoring tools
Metrics tool
Logging tools
APM tools
Infrastructure provisioning
tools
Security tools
Database management
tools
Testing tools
….
Not responsible for
*
*Unless their product belongs in the blue

15. We built tools to
automate our software
release process
https://secure.flickr.com/photos/lindseygee/5894617854/

16. Deployment service
No downtime
deployments
Health checking
Versioned artifacts
and rollbacks

17. Automated actions and
transitions; from check-
in to production
Development benefits:
• Faster
• Safer
• Consistent &
Standardized
• Visualization of the
process
Pipelines

18. Every year we survey our
software developers and in 2014
results found only one
development tool/service could
be correlated statistically with
happier developers:
Our pipelines service!

19. continuous delivery
==
happier developers!
https://www.flickr.com/photos/cannnela/4614340819/

20. AWS Code* Services
AWS CodePipeline AWS CodeDeploy AWS CodeCommit

21. AWS Code Services
Commit Build Test Production
Software Release Steps:

22. AWS Code Services
Commit Build Test Production
AWS CodeCommit
Software Release Steps:

23. AWS Code Services
Commit Build Test Production
Third Party
Tooling
Third Party
Tooling
Software Release Steps:

24. AWS Code Services
Commit Build Test Production
AWS CodeDeploy
Software Release Steps:

25. AWS Code Services
Commit Build Test Production
AWS CodeDeploy
Software Release Steps:
EC2 On-Prem

26. AWS Code Services
Commit Build Test Production
AWS CodePipeline
Software Release Steps:

27. AWS Code Services
Commit Build Test Production
AWS CodeCommit
AWS CodePipeline
AWS CodeDeployThird Party
Tooling
Third Party
Tooling
Software Release Steps:

28. Continuous delivery service for fast and
reliable application updates
Model and visualize your software release
process
Builds, tests, and deploys your code every time
there is a code change
Integrates with third-party tools and AWS
AWS CodePipeline

29. Source
Source
GitHub
Build
JenkinsOnEC2
Jenkins
Deploy
JavaApp
Elastic Beanstalk
Pipeline
Stage
Action
Transition
CodePipeline
MyApplication

30. Source
Source
GitHub
Build
JenkinsOnEC2
Jenkins
Deploy
JavaApp
Elastic Beanstalk
NotifyDevelopers
Lambda
CodePipeline
MyApplication
Parallel actions

31. Source
Source
GitHub
Build
JenkinsOnEC2
Jenkins
Deploy
JavaApp
Elastic Beanstalk
NotifyDevelopers
Lambda
TestAPI
Runscope
CodePipeline
MyApplication
Sequential actions

32. Build
JenkinsOnEC2
Jenkins
Staging-Deploy
JavaApp
Elastic Beanstalk
Prod-Deploy
JavaApp
Elastic Beanstalk
QATeamReview
Manual Approval
CodePipeline
MyApplication
Manual Approvals
Review

33. 8. Retrieve build artifact
EC2 instance
CodePipeline
2. Store source artifact
Source
Source
GitHub
Build
JenkinsOnEC2
Jenkins
Deploy
JavaApp
Elastic Beanstalk
Source Artifact
S3
Build Artifact
S3
5. Get source artifact
1. Get Changes
6. Store build artifact
3. Poll for Job
4. Acknowledge Job
7. Put Success
9. Deploy build artifact
Elastic Beanstalk
Web container
Java App
MyApplication

34. AWS Code Pipeline service integrations
Source Invoke Logic Deploy
AWS Elastic Beanstalk
Amazon S3 AWS CodeDeployAWS Lambda
AWS CodeCommit
AWS OpsWorks
AWS CloudFormation

35. We have a strong partner list, and it’s growing
Source Build Test Deploy

36. Extend AWS CodePipeline Using Custom Actions
Update tickets Provision resources
Update dashboards
Mobile testing
Send notifications Security scan

37. Build & test your
application
https://secure.flickr.com/photos/spenceyc/7481166880

38. Building Your Code
“Building” code typically refers to languages that
require compiled binaries:
• .NET languages: C#, F#, VB.net, etc.
• Java and JVM languages: Java, Scala,
JRuby
• Go
• iOS languages: Swift, Objective-C
We also refer to the process of creating Docker
container images as “building” the image. EC2

39. No Building Required!
Many languages don’t require building. These
are considered interpreted languages:
• PHP
• Ruby
• Python
• Node.js
You can just deploy your code!
EC2

40. Testing Your Code
Testing is both a science and an art form!
Goals for testing your code:
• Want to confirm desired functionality
• Catch programming syntax errors
• Standardize code patterns and format
• Reduce bugs due to non-desired application
usage and logic failures
• Make applications more secure

41. Deploying your
applications
https://secure.flickr.com/photos/simononly/15386966677

42. Automates code deployments to any instance
Handles the complexity of updating your
applications
Avoid downtime during application deployment
Rollback automatically if failure detected
Deploy to Amazon EC2 or on-premises
servers, in any language and on any operating
system
Integrates with third-party tools and AWS
AWS CodeDeploy

43. appspec.yml Example
version:
0.0
os:
linux
files:
-­‐
source:
/
destination:
/var/www/html
permissions:
-­‐
object:
/var/www/html
pattern:
“*.html”
owner:
root
group:
root
mode:
755
hooks:
ApplicationStop:
-­‐
location:
scripts/deregister_from_elb.sh
BeforeInstall:
-­‐
location:
scripts/install_dependencies.sh
ApplicationStart:
-­‐
location:
scripts/start_httpd.sh
ValidateService:
-­‐
location:
scripts/test_site.sh
-­‐
location:
scripts/register_with_elb.sh

44. appspec.yml Example
version:
0.0
os:
linux
files:
-­‐
source:
/
destination:
/var/www/html
permissions:
-­‐
object:
/var/www/html
pattern:
“*.html”
owner:
root
group:
root
mode:
755
hooks:
ApplicationStop:
-­‐
location:
scripts/deregister_from_elb.sh
BeforeInstall:
-­‐
location:
scripts/install_dependencies.sh
ApplicationStart:
-­‐
location:
scripts/start_httpd.sh
ValidateService:
-­‐
location:
scripts/test_site.sh
-­‐
location:
scripts/register_with_elb.sh
• Remove/add instance to ELB
• Install dependency packages
• Start Apache
• Confirm successful deploy
• More!
• Send application files to one
directory and configuration
files to another
• Set specific permissions on
specific directories & files

45. v2 v2 v2 v2 v2 v2
one at a time
half at a time
all at once
v2 v2 v2 v1 v1 v1
v2 v1 v1 v1 v1 v1 Agent Agent
Dev Deployment group
OR
Prod Deployment group
Agent
AgentAgent
Agent Agent
Agent
Choose Deployment Speed and Group

46. Building your application
development release
pipeline
https://www.flickr.com/photos/seattlemunicipalarchives/12504672623/

47. DEMO!

48. General Best Practices used by Amazon Developers
• CI/CD is a MUST!
• Commit frequently
• Builds on every commit
• Build once in a given execution flow
• Deploy to a running environment for further testing

49. General Best Practices used by Amazon Developers
• CI/CD is a MUST!
• Commit frequently
• Builds on every commit
• Build once in a given execution flow
• Deploy to a running environment for further testing
• Everything that is code (application, infrastructure, documentation)
goes into a repository
• If its not in a repository, it doesn’t go into Production environments!

50. General Best Practices used by Amazon Developers
• CI/CD is a MUST!
• Commit frequently
• Builds on every commit
• Build once in a given execution flow
• Deploy to a running environment for further testing
• Everything that is code (application, infrastructure, documentation)
goes into a repository
• If its not in a repository, it doesn’t go into production environments!
• Start with continuous delivery (“gated” promotion) and build up to
continuous deployment once evidence of a high-level of excellence in
testing is clear

51. General Best Practices used by Amazon Developers
• CI/CD is a MUST!
• Commit frequently
• Builds on every commit
• Build once in a given execution flow
• Deploy to a running environment for further testing
• Everything that is code (application, infrastructure, documentation)
goes into a repository
• If its not in a repository, it doesn’t go into production environments!
• Start with continuous delivery (“gated” promotion) and build up to
continuous deployment once evidence of a high-level of excellence in
testing is clear
• Deploy to canaries, test, deploy to an AZ, test, deploy to a Region,
test

52. General Best Practices used by Amazon Developers (cont.)
• Code Reviews are one of the best mechanisms for “good” code:
• Does this code look clean and can someone else understand it?
• Is the design of it meeting the expectations of its needs?
• Are there better/easier ways to do this same thing?

53. General Best Practices used by Amazon Developers (cont.)
• Code Reviews are one of the best mechanisms for “good” code:
• Does this code look clean and can someone else understand it?
• Is the design of it meeting the expectations of its needs?
• Are there better/easier ways to do this same thing?
• Style checkers
• Will someone else in the company be able to update/fix/maintain this code?

54. General Best Practices used by Amazon Developers (cont.)
• Code Reviews are one of the best mechanisms for “good” code:
• Does this code look clean and can someone else understand it?
• Is the design of it meeting the expectations of its needs?
• Are there better/easier ways to do this same thing?
• Style checkers
• Will someone else in the company be able to update/fix/maintain this code?
• Auto-rollbacks can be the quickest recovery mechanism after failure
• Rollback first, then debug what went wrong with logs/graphs/etc.

55. General Best Practices used by Amazon Developers (cont.)
• Code Reviews are one of the best mechanisms for “good” code:
• Does this code look clean and can someone else understand it?
• Is the design of it meeting the expectations of its needs?
• Are there better/easier ways to do this same thing?
• Style checkers
• Will someone else in the company be able to update/fix/maintain this code?
• Auto-rollbacks can be the quickest recovery mechanism after failure
• Rollback first, then debug what went wrong with logs/graphs/etc.
• Thorough dashboards
• What is happening now?
• What ”normal” looks like typically over some period of time?
• What do I do if this graph looks wrong/an alarm has been triggered?
• What events can I correlate with a move in a graph?

56. Code* Tips and Tricks
• All Code* products can(and should) be provisioned and managed
with AWS CloudFormation!
• You could literally store the CloudFormation templates that provision
your Code* resources in CodeCommit and update them via
CodePipeline (It’s like Code* Inception!)
• Deep integration with IAM. You can assign permissions on who can
commit code, approve manual approvals, deploy to certain
deployment groups and more!
• Integrate with AWS Lambda to do almost anything:
• CodeCommit has Repository Triggers
• CodeDeploy has Event Notifications
• CodePipeline has native Lambda invoke
AWS CodePipeline AWS CodeDeploy AWS CodeCommit

57. FIN, ACK
We’ve quickly run through the AWS Code* services today
at a high level and have talked a little bit about how we do
things here at Amazon:
• Automate CI & CD to reduce errors from manual testing
and deployment, increase release velocity, minimize
errors that make it out in front of your users
• Store everything as code
• > 20 talks here at re:Invent about DevOps so check
them out! (on the web)

58. Some of the other DevOps talks this week!
o DEV303 - Deploying and Managing .NET Pipelines and Microsoft Workloads
o DEV310 - DevOps on AWS: Choosing the Right Software Deployment Technique
o DEV313 - Infrastructure Continuous Deployment Using AWS CloudFormation
o DEV403 - DevOps on AWS: Advanced Continuous Delivery Techniques
o SVR307 - Application Lifecycle Management in a Serverless World

59. aws.amazon.com/devops

60. AWS DevOps Blog

61. Thank you!
DEV201