The document summarizes Tiago Henriques' presentation on continuous deployment at the AWS Summit 2013 in Tel Aviv. It describes how Tiago discussed using AWS services like CloudFormation to automate deployments and provision test environments on demand for low cost. It also outlines CloudOn's development and test deployment model of using "All In One" instances that allow developers to easily deploy any code branch for integration testing.

1. AWS Summit 2013 Tel Aviv
Oct 16 – Tel Aviv, Israel
Continuous Deployment
Tiago Henriques
Solutions Architect, Amazon Web Services

2. CONTINUOUS
INTEGRATION

3. DEVELOPER

4. SOURCE CODE
REPOSITORY

5. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER

6. SOURCE CODE
REPOSITORY
CONTINUOUS
INTEGRATION SERVER
PROJECT MANAGEMENT
SERVER

7. SOURCE CODE
REPOSITORY
CONTINUOUS
INTEGRATION SERVER
PROJECT MANAGEMENT
SERVER
PICK
TASKS

8. SOURCE CODE
REPOSITORY
SUBMIT
CODE
CONTINUOUS
INTEGRATION SERVER
PROJECT MANAGEMENT
SERVER

9. SOURCE CODE
REPOSITORY
CONTINUOUS
INTEGRATION SERVER
SCHEDULE
BUILD
PROJECT MANAGEMENT
SERVER

10. SOURCE CODE
REPOSITORY
CONTINUOUS
INTEGRATION SERVER
RECURRENT
BUILDS
PROJECT MANAGEMENT
SERVER

11. SOURCE CODE
REPOSITORY
CONTINUOUS
INTEGRATION SERVER
CODE
FETCH
PROJECT MANAGEMENT
SERVER

12. SOURCE CODE
REPOSITORY
CONTINUOUS
INTEGRATION SERVER
CODE QUALITY
TESTS
PROJECT MANAGEMENT
SERVER
TEST
RESULTS

13. SOURCE CODE
REPOSITORY
CONTINUOUS
INTEGRATION SERVER
BUILD OUTPUT
PROJECT MANAGEMENT
SERVER

14. SOURCE CODE
REPOSITORY
DOCS
CONTINUOUS
INTEGRATION SERVER
BINARIES
& PACKAGES
PROJECT MANAGEMENT
SERVER

16. SOURCE CODE
REPOSITORY
BUILDS
CONTINUOUS
INTEGRATION SERVER
DNS
PROJECT
MANAGEMENT SERVER

17. PAIN POINTS
•
•
•
•
•
UNIT TESTS INCOMPLETE
MOCK TESTS MAINTENANCE
EXPENSIVE TEST ENVIRONMENT
TEST ENVIRONMENT ≠ PRODUCTION
DEPLOYMENT CYCLES

18. ON-DEMAND
ELASTIC
PAY AS YOU GO

19. =
PROGRAMMABLE
PLATFORM

23. IF YOU CAN PROGRAM IT
YOU CAN AUTOMATE IT

24. AWS
CLOUDFORMATION
STACK-BASED DEPLOYMENT
SERVICE

25. CLOUDFORMATION
TEMPLATE

27. {
"Description" : "Create RDS with username and password",
"Resources" : {
"MyDB" : {
"Type" : "AWS::RDS::DBInstance",
"Properties" : {
"AllocatedStorage" : "500",
"DBInstanceClass" : "db.m1.small",
"Engine" : "MySQL",
"EngineVersion" : "5.5",
"MasterUsername" : "MyName",
"MasterUserPassword" : "MyPassword"
}
}
}
}

28. "AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
"yum" : {
"mysql"
: [],
"mysql-server" : [],
"httpd"
: [],
"php"
: [],
"php-mysql"
: []
}
},
"sources" : {
"/var/www/html" :
"https://s3.amazonaws.com/my-builds/build-v4.zip"
}
}

29. {
"Parameters" : {
"KeyName" : {
"Description" : "Name of an existing EC2
KeyPair to enable SSH
access to the instance",
"Type" : "String"
}
},
}

30. PROCEDURAL
DEFINITION
KNOWN
CONFIGURATION
Create it programmatically
Store stack configuration in
source control
CLOUDFORMATION
TEMPLATE
PARAMETER
DRIVEN
Dynamic and user-driven
templates
COLLABORATION
Share templates with ease
as just files

32. APPLICATION
VERSIONS
+
INFRASTRUCTURE
VERSIONS

33. CLOUDFORMATION
TEMPLATE

38. HTTP://SORCERY.SMUGMUG.COM/

39. TEST ENVIRONMENTS

40. 30,000 REQUESTS / SECOND
1 TB TRAFFIC / DAY

41. “…AWS seemed to be the best solution available to allow a
small, independent company to rapidly develop and test a
completely new infrastructure, and host it.
We also loved the flexibility that AWS allowed us, when
spinning up smaller test environments, for beta trials, QA,
localization, and during development. The low initial cost
was also crucial.”
Alex Evans, CTO

42. LOAD TESTING

48. USING AMAZON EC2 TO SIMULATE
2.4 MILLION PLAYERS

49. CONTINUOUS
DEPLOYMENT
SMALL, FREQUENT CHANGES
CONSTANTLY INTEGRATING INTO
PRODUCTION

51. KEY = ITERATION

52. ITERATION
=
MODIFY THE SYSTEM TO BETTER
MEET THE EXPECTATIONS OF
YOUR USERS

53. DEPLOYMENTS AT
AMAZON.COM
11.6s
1,079
10,000
30,000
Mean time
between
deployments
(weekday)
Max number of
deployments in a
single hour
Mean number of
hosts
simultaneously
receiving a
deployment
Max number of
hosts
simultaneously
receiving a
deployment

54. SOFTWARE DEPLOY
≠
PRODUCT LAUNCH

56. 1.5 BILLION PAGE VIEWS
$83 MILLION IN TRANSACTIONS
4.2 MILLION ITEMS SOLD
OCTOBER 2012

57. 30 DEPLOYS PER DAY
1 DEPLOY EVERY 20 MINUTES

58. "Production is truly the only place you
can validate your code."

62. AWS OPSWORKS
INTEGRATED APPLICATION
MANAGEMENT

64. 14 BILLION REQUESTS/MONTH
50 000 DATABASE UPDATES / SEC
NO CACHE

65. A / B TESTING

70. DATA-DRIVEN
ARCHITECTURES

71. METRICS @ETSY

72. COST-ORIENTED
ARCHITECTURES

73. PHP+APACHE+VARNISH
NGINX+NODEJS

75. CONTINUOUS
DEPLOYMENT
CONTINUOUS
INTEGRATION

76. CONTINUOUS
DEPLOYMENT
=
CONTINUOUS
EXPERIMENTATION

77. CONTINUOUS
DEPLOYMENT
=
CONTINUOUS
IMPROVEMENT

78. INNOVATE

79. « Want to increase innovation?
Lower the cost of failure »
Joi Ito

80. SPEED AND AGILITY
“ON-PREMISE”
Experiment
Infrequently
Experiment
Often
Failure is
expensive
Fail quickly at
a low cost
Less
Innovation
More
Innovation

81. CloudOn Dev / Test Flows
AWS IL Conference
Oct 16 2013

82. What is CloudOn?
CloudOn is a versatile productivity
solution that allows you to connect with
cloud storage providers to create,
review, edit and share your Micosoft
Office files across different devices.
Proprietary and Confidential | 82

83. The CloudOn Production Environment
Provisioning
Storage
Social
Sessions DB
File Sharing
API
RDS
Proprietary and Confidential | 83

84. Production Deployment Flow

Bootsrapping
Launch Instances
install 3rd party infra (Java, Tomcat, etc)
Configure roles

Deploy CloudOn Java services



(Not using OpsWorks yet)
Proprietary and Confidential | 84

85. Dev / Test Deployment Needs
Developer Needs



Push-button deploy by developers
Deploy any branch
Several deployments in parallel
Automated Regression Needs



Deploy specific release candidate combos
Run parallel regressions
Low Cost
Proprietary and Confidential | 85

86. The CloudOn Dev/Test Deployment Model
All In One
Proprietary and Confidential | 86

87. All In One (AIO) - Deploy



A single instance hosting all CloudOn services
Launches via Jenkins Job
All In One
Allows to override defaults:







Branch and build
Instance type
Regions
Settings
Tagged by the developer name
Each CloudOn service runs on a different port, no
collision
Allows re-deploy on existing AIO
Proprietary and Confidential | 87

88. All In One – External Services
What to do with the external, AWS services?






RDS
DynamoDB
ElastiCache
RDS
SQS
SES
Proprietary and Confidential | 88

89. All In One – SQS, DynamoDB, SES




Using a shared dev account - SQS, DynamoDB, SES
New Queue, Tables created for each AIO
Keyed by the instance private IP
Old Queues and Tables (from obsolete AIOs) deleted upon
creation of the newest AIO
Proprietary and Confidential | 89

90. All In One – RDS, Memcached




Using a local mySQL, memcached on the machine
Allows each instance to have its own data
Allows exploring private schema changes
(No cost)
Proprietary and Confidential | 90

91. All In One – Cost




Medium instance (default, can be changed)
Most AIOs are automatically destroyed at midnight
(unless requested not to at creation time)
Weekly report on long-running AIOs
(still find rogue AIOs running for too long…)
Proprietary and Confidential | 91

92. All In One – Additional Benefits
•
•
•
•
•
•
Push-button deploy by developers
On demand
Easy end-to-end integration test on AWS
Any branch and version
Allows operational testing
Allows sandboxing for QA, partners
Proprietary and Confidential | 92

93. Thanks!
Proprietary and Confidential | 93