With AWS companies now have the ability to develop and run their applications with speed and flexibility like never before. Working with an infrastructure that can be 100% API driven enables businesses to use lean methodologies and realize these benefits. This in turn leads to greater success for those who make use of these practices. In this session we'll talk about some key concepts and design patterns for Continuous Deployment and Continuous Integration, two elements of lean development of applications and infrastructures.

1. © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
Continuous Integration and
Deployment on AWS
Martin Elwin
Manager, Solutions Architecture

2. CONTINUOUS
INTEGRATION

3. DEVELOPER

4. SOURCE CODE
REPOSITORY

5. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER

6. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER

7. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
PICK
TASKS

8. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
SUBMIT
CODE

9. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
CODE
FETCH

10. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
BUILD OUTPUT

11. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
DOCS
BINARIES
& PACKAGES

12. SOURCE CODE
REPOSITORY
DNS
CONTINUOUS
INTEGRATION SERVER
PROJECT
MANAGEMENT SERVER
BUILDS

13. PAIN POINTS
• UNIT TESTS INCOMPLETE
• MOCK MAINTENANCE
• EXPENSIVE TEST ENVIRONMENT
• TEST ENVIRONMENT ≠ PRODUCTION

14. ON-DEMAND
PAY AS YOU GO
ELASTIC

17. = PROGRAMMABLE PLATFORM

18. …
ec2.runInstances(new RunInstancesRequest()
.withImageId("ami-4b814f22")
.withInstanceType("m1.small")
.withMinCount(1)
.withMaxCount(1)
.withKeyName("mykey")
.withSecurityGroups("httpssh"));
…
JAVA SDK

19. …
var runInstanceRequest = new
RunInstancesRequest()
{
ImageId = "ami-4b814f22",
InstanceType = "m1.small",
MinCount = 1,
MaxCount = 1,
KeyName = "mykey"
};
runInstanceRequest.SecurityGroup.Add("httpssh");
ec2.RunInstances(runInstanceRequest);
…
.NET SDK

20. …
ec2.run_instances(
'ami-4b814f22',
key_name='mykey',
instance_type='m1.small',
min_count=1,
max_count=1,
security_groups=['httpssh'])
…
PYTHON SDK

21. …
ec2.instances.create(
:image_id => 'ami-4b814f22',
:instance_type => 'm1.small',
:count => 1,
:security_groups => 'httpssh',
:key_pair =>
ec2.key_pairs['mykey'])
…
RUBY SDK

22. …
$ec2 -> run_instances(
'ami-4b814f22', 1, 1,
array(
'InstanceType' => 'm1.small',
'KeyName' => 'mykey',
'SecurityGroup' => 'httpssh')
);
…
PHP SDK

23. …
var params = {
ImageId: 'ami-4b814f22',
InstanceType: 'm1.small',
MinCount: 1,
MaxCount: 1,
SecurityGroups: ['httpssh']
};
ec2.runInstances(params, function(err, res) {
…
});
…
JAVASCRIPT SDK

24. aws ec2 run-instances
--image-id ami-4b814f22
--min-count 1
--max-count 1
--key-name mykey
--security-groups httpssh
UNIFIED CLI

25. New-EC2Instance
-ImageId ami-4b814f22
-MinCount 1
-MaxCount 1
-KeyName mykey
-SecurityGroupId sg-9cf9e5d9
-InstanceType m1.small
POWERSHELL CLI

26. https://ec2.amazonaws.com/
?Action=RunInstances
&ImageId=ami-4b814f22
&MaxCount=1
&MinCount=1
&SecurityGroup.1=httpssh
&KeyName=mykey
&<AUTHPARAMS>
AWS REST API

28. IF YOU CAN PROGRAM IT
YOU CAN AUTOMATE IT

33. AWS
CLOUDFORMATION
STACK-BASED
DEPLOYMENT SERVICE

34. AWS CLOUDFORMATION
TEMPLATE

35. {
"Description" : "Create a small instance",
"Resources" : {
"MyInstance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"ImageId" : "ami-4b814f22",
"KeyName" : "mykey",
"InstanceType" : "m1.small",
"SecurityGroups" : ["httpssh"]
}
}
}
}

36. CLOUDFORMATION
TEMPLATE
DECLARATIVE
DEFINITION
Create it programmatically
KNOWN
CONFIGURATION
Store stack configuration in
source control
PARAMETER
DRIVEN
Dynamic and user-driven
templates
COLLABORATION
Share templates with ease
as just files

38. APPLICATION
VERSIONS
+
INFRASTRUCTURE
VERSIONS

39. AWS
CLOUDFORMATION
TEMPLATE

40. Continuous Deployment at Viaplay

41. • Viaplay is Nordic’s leading online video service for TV, sport,
film and kids' content.
• The service is available on PC, Mac, Android and iOS mobile
phones, tablets, the Viaplay box, LG Smart TV, Samsung Smart
TV, Philips Smart TV, Panasonic Smart TV, Xbox 360,
PlayStation®3 and PlayStation®4.

42. Preparations for the Olympics
• New content / video / subscriber management platform
• API:s completely rebuilt in Node.js in a service oriented
architecture
• All clients rebuilt (Web, Smartphones, Smart TV:s, Gaming
consoles, Set Top Box)
• Moved hosting of API:s to AWS

43. System Setup
• Almost everything described in CloudFormation
templates: Network (VPC), Load Balancers, Cache clusters,
DNS entries, Autoscaling groups
• The same templates for all different environments
• Every application has its own CloudFormation stack with DNS
entries, Elastic Load Balancers and an Auto Scaling Group

44. Auto Scaling
• An ASG contains a variable number of instances running the
same AMI
• To enable auto scaling, the AMI must start automatically
• Application servers starts with our custom AMI containing
everything except the application build

45. Auto Scaling
• Build and environment are set in tags and UserData. UserData
is run at instance startup, tags are readable from the instance
but more important clearly visible in the console.
• When an EC2 instance is launched it fetches the build from S3,
installs and starts it
• As soon as the ELB health check is OK, the instance gets traffic

46. Deploy
• Autoscaling has already made us create AMIs that starts the
application at instance startup
• Deploy is done by changing parameters to the stack
• Setting new UserData on the ASG/EC2 launch config requires
exchange of instances. CloudFormation does that for us with
the Rolling Update policy

47. Lessons learnt
• Rolling update takes time and is practically unstoppable
during update. We are currently rewriting this so that we start
a new ASG that completely replaces the old one
• To be able to do a change in a stack, your IAM user needs to
have the rights to change all resources in the template. All
developers must be able to deploy their code, but we do not
want all developers to be Power Users.

48. Custom Admin Application
• Our admin application enables CloudFormation updates
without giving Power User rights to all developers
• We can keep track of who has done what in an easy way
• The application also makes sure that deploys are done in
exactly the same way every time
• Built on top of the AWS Javascript SDK

49. Custom Admin Application
• Our admin application enables CloudFormation updates
without giving Power User rights to all developers
• We can keep track of who has done what in an easy way
• The application also makes sure that deploys are done in
exactly the same way every time
• Built on top of the AWS Javascript SDK

50. Deploy From a Web Interface
• Build OK from Jenkins -> copied to S3 builds bucket
• Deploy tool lists all files from the builds bucket

51. Deploy From a Web Interface
• Write a short description of the deploy for the release email

52. Deploy From a Web Interface
• The admin tool updates the CloudFormation stack with the
new build parameter
• The deploy notifier application sends an email with
information on all commits done between the current and the
new build

54. Deploy From a Web Interface
• If everything looks good on stage - click ”thumbs up” to copy
the build to the production bucket, else ”thumbs down” to
reject the build

55. Ready for the Olympics
• Everything deployed during the winter
• Last user migrated in January
• Opening ceremony at Friday Feb 7

56. Ready for the Olympics
• 39 deploys during the week starting Feb 3
• 11 deploys in 6 different applications on the day of the
opening ceremony
• All went fine

57. Thanks!
johan.waernulf@viaplay.com

58. CONTINUOUS
DEPLOYMENT
SMALL, FREQUENT CHANGES
CONSTANTLY INTEGRATING INTO
PRODUCTION

59. KEY = ITERATION

60. ITERATION
=
MODIFY THE SYSTEM TO BETTER
MEET THE EXPECTATIONS OF
YOUR USERS

61. 11.6s
Mean time
between
deployments
(weekday)
1,079
Max number of
deployments in a
single hour
10,000
Mean number of
hosts
simultaneously
receiving a
deployment
30,000
Max number of
hosts
simultaneously
receiving a
deployment
DEPLOYMENTS AT
AMAZON.COM

62. SOFTWARE DEPLOY
≠
PRODUCT LAUNCH

64. 1.8 BILLION PAGE VIEWS
$109 MILLION IN SALES
5.5 MILLION ITEMS SOLD

65. 30 DEPLOYS PER DAY
1 DEPLOY EVERY 20 MINUTES

66. "Production is truly the only place
you can validate your code."

67. RUN-TIME CONFIGURATION OF ACTIVE FEATURES

68. AWS OPSWORKS
INTEGRATED APPLICATION
MANAGEMENT

70. 14 BILLION REQUESTS/MONTH
50 000 DATABASE UPDATES/SEC

71. A / B TESTING

74. DATA-DRIVEN
ARCHITECTURES

75. METRICS @ETSY

76. CONTINUOUS
INTEGRATION
CONTINUOUS
DEPLOYMENT

77. CONTINUOUS
DEPLOYMENT
=
CONTINUOUS
EXPERIMENTATION

78. CONTINUOUS
DEPLOYMENT
=
CONTINUOUS
IMPROVEMENT

79. « Want to increase innovation?
Lower the cost of failure »
Joi Ito

80. SPEED AND AGILITY
Experiment
Often
Fail quickly at
a low cost
More
Innovation
Experiment
Infrequently
Failure is
expensive
Less
Innovation
“ON-PREMISES”

81. wow
so deploy
much innovate
such agility
very continuous

82. © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
Thanks!
Continuous Integration and
Deployment on AWS
Martin Elwin
Manager, Solutions Architecture