This document discusses automating the application lifecycle using infrastructure as code principles. It demonstrates building infrastructure like VPCs and databases automatically with CloudFormation templates. It then shows deploying application components like load balancers and servers from templates. It discusses updating applications by building new server images from code changes rather than modifying existing servers. This allows deploying updates instantly by replacing server instances. The talk concludes that automating infrastructure and deployments in this way allows integrating new applications or changes much faster.

1. Automating the application lifecycle
19 mars 2015 . #TIAD . @tiadparis

2. # TIAD@ tiadparis
Who am I?
2
Laurent Bernaille @d2si
• Linux background
• Cloud enthousiast
• Opensource advocate
• Love discovering, building (and breaking…) new things
• Passionate about the ongoing IT transformations
@lbernail

3. # TIAD@ tiadparis
Story behind this talk/demo
3
A classic company with a « standard » on-premise IT
New business objectives in a competitive space: IOT
Creation of small, independant start-up teams
Use of Amazon Web Services for agility and scalability
An opportunity to use a new application delivery process

4. # TIAD@ tiadparis
Automating the application lifecyle: Objectives
4
Integrate a new application in hours instead of days
Create a new environment in minutes instead of month
Deploy a new version of the application in minutes instead of hours

5. # TIAD@ tiadparis
What it was (is?) like
5
DEV
Version
Control
System Continuous
Integration
Analyse code
Build Repository
Push code to VCS
Traditional Continuous integration
• Build a binary application artifact
• WAR, JAR, RPM, DEB, ZIP
Artifact
Traditional Deployment
• Build environments
• Deploy the artifact in environments
OPS
Configured server(s)
Staging
Configured server(s)
Production
We can do better!

6. # TIAD@ tiadparis
Demo outline
6
1. Infrastructure build
1. Creation of application back-ends
2. Deployment of the application
3. New release

7. # TIAD@ tiadparis
Step 1: Build infrastructure automatically
7
Build a full environment from scratch
• Subnets
• Routing tables
• Technical services

8. # TIAD@ tiadparis
Step 1: Build infrastructure automatically
7
CloudFormation
Build a full environment from scratch
• Subnets
• Routing tables
• Technical services
Cloudformation
• Describe infrastructure components
• Build them
• « puppet » for infrastructure
• AWS only: see also Terraform / Heat

9. # TIAD@ tiadparis
Step 1: Build infrastructure automatically
7
CloudFormation
Build a full environment from scratch
• Subnets
• Routing tables
• Technical services
eu-west-1a
Public subnets
Private subnets
eu-west-1b
Public subnets
Private subnets
Cloudformation
• Describe infrastructure components
• Build them
• « puppet » for infrastructure
• AWS only: see also Terraform / Heat

10. # TIAD@ tiadparis
Step 1: Build infrastructure automatically
7
CloudFormation
NAT NATBastion
Build a full environment from scratch
• Subnets
• Routing tables
• Technical services
eu-west-1a
Public subnets
Private subnets
eu-west-1b
Public subnets
Private subnets
Cloudformation
• Describe infrastructure components
• Build them
• « puppet » for infrastructure
• AWS only: see also Terraform / Heat

11. # TIAD@ tiadparis
Step 1: Build infrastructure automatically
7
CloudFormation
NAT NATBastion
Build a full environment from scratch
• Subnets
• Routing tables
• Technical services
eu-west-1a
Public subnets
Private subnets
eu-west-1b
Public subnets
Private subnets
Why automate these rare actions?
• Avoid errors
• Create identical environments (Production, Staging) easily
• Ability to deploy somewhere else very quickly
• Not so rare
• Easier to update
Cloudformation
• Describe infrastructure components
• Build them
• « puppet » for infrastructure
• AWS only: see also Terraform / Heat

12. # TIAD@ tiadparis
Under the hood
8
NAT NATBastion
eu-west-1a
Public subnets
Private subnets
eu-west-1b
Public subnets
Private subnets
"VPC" : {
"Type" : "AWS::EC2::VPC",
"Properties" : {
"CidrBlock" : "10.100.0.0/16"
}
”PublicSubnet1" : {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"VpcId" : { "Ref" : "VPC" },
"AvailabilityZone" : "eu-west-1a"
"CidrBlock” : "10.100.1.0/25”
}
}
"NatInstance1" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"ImageId" : "ami-6e7bd919",
"InstanceType" : "t2.micro"
"SourceDestCheck" : "False",
"UserData” :
{ "Fn::Base64" : { "Fn::Join" : [”n", [
"#!/bin/bash",
"echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf",
"sysctl -p,
"iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE",
"iptables-save > /etc/sysconfig/iptables"
]]}}
}
}

13. # TIAD@ tiadparis
Step 2: Build application components
9
Build all the backends
• Databases
• Buckets
• Cache servers
• Queues & Topics
NAT NATBastion
eu-west-1a
Public subnets
Private subnets
eu-west-1b
Public subnets
Private subnets

14. # TIAD@ tiadparis
Step 2: Build application components
9
CloudFormation
Build all the backends
• Databases
• Buckets
• Cache servers
• Queues & Topics
NAT NATBastion
eu-west-1a
Public subnets
Private subnets
eu-west-1b
Public subnets
Private subnets
DynamoDB
DynamoDB
• AWS NoSQL database
• « Cassandra as a service »

15. # TIAD@ tiadparis
Step 2: Build application components
9
CloudFormation
Build all the backends
• Databases
• Buckets
• Cache servers
• Queues & Topics
NAT NATBastion
eu-west-1a
Public subnets
Private subnets
eu-west-1b
Public subnets
Private subnets
Why separate from servers where the application run?
• Different lifecycle
• Can be shared between releases
DynamoDB
DynamoDB
• AWS NoSQL database
• « Cassandra as a service »

16. # TIAD@ tiadparis
Difficulty: managing variables
10
VPC Addresses
Subnets
Instance types
Infra template

17. # TIAD@ tiadparis
Difficulty: managing variables
10
VPC Addresses
Subnets
Instance types
Infra template
Parameters

18. # TIAD@ tiadparis
Difficulty: managing variables
10
VPC Addresses
Subnets
Instance types
Infra template
Parameters
Backend template
VPC Id
Subnet Ids
DB Name

19. # TIAD@ tiadparis
Difficulty: managing variables
10
VPC Addresses
Subnets
Instance types
Infra template
Parameters
Backend template
VPC Id
Subnet Ids
DB Name

20. # TIAD@ tiadparis
Difficulty: managing variables
10
VPC Addresses
Subnets
Instance types
Infra template
Parameters
Backend template
VPC Id
Subnet Ids
DB Name
?
?

21. # TIAD@ tiadparis
Difficulty: managing variables
10
VPC Addresses
Subnets
Instance types
Infra template
Parameters
Backend template
VPC Id
Subnet Ids
DB Name
?
?
Wrapper to manage inputs/outputs
Outputs

22. # TIAD@ tiadparis
Step 3: Deploy application
11
Deploy application
• Load-balancers
• Servers
• DNS Alias
NAT NATBastion
eu-west-1a
Public subnets
Private subnets
eu-west-1b
Public subnets
Private subnets
DynamoDB

23. # TIAD@ tiadparis
Step 3: Deploy application
11
CloudFormation
Deploy application
• Load-balancers
• Servers
• DNS Alias
NAT NATBastion
eu-west-1a
Public subnets
Private subnets
eu-west-1b
Public subnets
Private subnets
DynamoDB
WEB WEB
Load-balancer included to allow deployment of several versions
Web server is built from an image with application pre-installed

24. # TIAD@ tiadparis
Difficulty: managing variables, again
12
VPC Addresses
Subnets
Instance types
Infra template Backend template
VPC Id
Subnet Ids
DB Name
Outputs
Application template
Instance Types
DB Name

25. # TIAD@ tiadparis
Difficulty: managing variables, again
12
VPC Addresses
Subnets
Instance types
Infra template Backend template
VPC Id
Subnet Ids
DB Name
Outputs
Application template
Instance Types
DB Name
WEB
DynamoDB
?

26. # TIAD@ tiadparis
Difficulty: managing variables, again
12
VPC Addresses
Subnets
Instance types
Infra template Backend template
VPC Id
Subnet Ids
DB Name
Outputs
Application template
Instance Types
DB Name
WEB
DynamoDB
?
"UserData” :
{ "Fn::Base64" : { "Fn::Join" : ["", [
"#!/bin/bashn",
"echo ", { "Ref" : "Properties" } ," >> /var/www/html/application.propertiesn”
]]}}
Properties
Properties

27. # TIAD@ tiadparis
Updating the application: the old-fashion way
13
DEV
Version
Control
System Continuous
Integration
Analyse code
Build Repository
Push code to VCS
Traditional Continuous integration
• Build a binary application artifact
• WAR, JAR, RPM, DEB, ZIP
Artifact
Traditional Deployment
• Use previously built environments
• Deploy the artifact
OPS
Configured server(s)
Staging
Configured server(s)
Production
Environments will drift

28. # TIAD@ tiadparis
Updating the application: with config management
14
Environments can still drift
Not suited for autoscaling
DEV
Version
Control
System Continuous
Integration
Analyse code
Build Repository
Push code to VCS
Traditional Continuous integration
• Build a binary application artifact
• WAR, JAR, RPM, DEB, ZIP
Artifact
Use config management tools
• Same recipes across environments
• Same artefact across environments
• Manual application deployment
OPS
Configured server(s)
Staging
Configured server(s)
Production

29. # TIAD@ tiadparis
Updating the application: in the cloud
15
Building can take a while
Some drift still possible
DEV
Version
Control
System Continuous
Integration
Analyse code
Build Repository
Push code to VCS
Traditional Continuous integration
• Build a binary application artifact
• WAR, JAR, RPM, DEB, ZIP
Artifact
Use config management tools
• Same recipes across environments
• Same artefact across environments
• Deploy application at server start
OPS
Configured server(s)
Staging
Configured server(s)
Production

30. # TIAD@ tiadparis
Updating the application
16
DEV
Version
Control
System Continuous
Integration
Analyse code
Build
Repository
Push code to VCS
New Continuous integration
• Build an application artifact
• Build a server image
Artifact
OPS
Staging Production

31. # TIAD@ tiadparis
Updating the application
16
DEV
Version
Control
System Continuous
Integration
Analyse code
Build
Repository
Push code to VCS
New Continuous integration
• Build an application artifact
• Build a server image
Artifact
OPS
Staging Production
Provision,
Config, deploy
Reference
templatesBuild
Application
templates

32. # TIAD@ tiadparis
Updating the application
16
DEV
Version
Control
System Continuous
Integration
Analyse code
Build
Repository
Push code to VCS
New Continuous integration
• Build an application artifact
• Build a server image
Artifact
OPS
Configured server(s)
Staging
Configured server(s)
Production
Provision,
Config, deploy
Reference
templatesBuild
Application
templates

33. # TIAD@ tiadparis
Demo
17
Push code
Integration
AWS images
AWS
Application
templates
Web hook
Packer
• Automate the creation of templates
• Developped by @mitchellh / @hashicorp

34. # TIAD@ tiadparis
Demo
17
Push code
Integration
AWS images
AWS
Application
templates
Web hook
Build
WEB WEB
Packer
• Automate the creation of templates
• Developped by @mitchellh / @hashicorp

35. # TIAD@ tiadparis
Packer
18
"builders": [{
"type": "amazon-ebs",
"region": ”eu-west-1",
"source_ami": "ami-f0b11187",
"instance_type": "t2.small",
"ssh_username": "ubuntu",
"ami_name": "demo-{{isotime "2006-01-02T15-04-05"}}"
}],
"provisioners": [
{
"type": "file",
"source": "/jenkins_workspace/site",
"destination": "/tmp/www"
},
{
"type": "shell",
"inline": [
"sudo apt-get update",
"sudo apt-get install -y apache2 php5 libapache2-mod-php5 php5-curl php5-mysql",
"sudo rm /var/www/html/*",
"sudo mv /tmp/www/* /var/www/html",
"sudo service apache2 restart"
]
}]

36. # TIAD@ tiadparis 19
Push code
Continuous
Integration
Application
templates
IntegrationPerspectives

37. # TIAD@ tiadparis 19
Production
Push code
Continuous
Integration
Application
templates
Integration
Prod DNS
Perspectives

38. # TIAD@ tiadparis 19
Production
Push code
Continuous
Integration
Application
templates
Integration
Prod DNS
Perspectives

39. # TIAD@ tiadparis 19
Production
Test
Push code
Continuous
Integration
Application
templates
Integration
Prod DNS
Perspectives

40. # TIAD@ tiadparis 19
Production
Test
Push code
Continuous
Integration
Application
templates
Integration
Prod DNS
Perspectives

41. # TIAD@ tiadparis 19
Production
Test
Push code
Continuous
Integration
Application
templates
Integration
Prod DNS
Perspectives

42. # TIAD@ tiadparis
Key take-aways
20
Everything can be automated
Very important change: Immutable servers
• New application artifact: Images / Containers
• Very challenging for organizations

43. # TIAD@ tiadparis
Thank you
@lbernail
Fork the code of this demo on github
https://github.com/lbernail/demo