My talk at FullStackFest, 4.9.2017. Become more familiar with managing infrastructure using Terraform, Packer and deployment pipeline. Code repository - https://github.com/antonbabenko/terraform-deployment-pipeline-talk

1. Terraform in deployment pipeline
by Anton Babenko

2. Hi!
I am Anton Babenko, and I enjoy:
● AWS & DevOps
● AWS User Group Norway and DevOpsDays Oslo organizer
● Solve problems
“Getting Started with Terraform”,
terraform-community-modules, Terraform modules generator
(Terrapin), and more…
https://github.com/antonbabenko https://www.linkedin.com/in/antonbabenko

3. 1. Become more familiar with managing infrastructure using CD pipeline
2. See scenarios of integrating Terraform and Packer
3. How to structure infrastructure code?
4. How to version your infrastructure between environments and make it DRY?
Goals of this talk

4. Do you know?
What is:
● Infrastructure as code?
● Deployment pipeline?
● Pipeline as code?

5. Featuring...
Write, plan, and create infrastructure as code Build automated machine images

7. Typical CI/CD pipeline
source: https://dzone.com/articles/what-is-continuous-delivery-pipeline

8. Where are infrastructure changes here?

9. CI/CD pipeline (CircleCI 2.0)

10. Structure - all-in-one vs split
~/all-in-one-repo/
├── packer # Packer configs
│ └── app.json
├── terraform # Terraform configs
│ ├── main.tf
│ └── terraform.tfvars
└── web # Application code
└── index.html
~/infra-repo/
├── packer # Packer configs
│ └── app.json
└── terraform # Terraform configs
├── main.tf
└── terraform.tfvars
~/app-repo/
└── web # Application code
└── index.html

11. Structure - evolving infrastructure repository
~/infra-repo/
├── packer # Packer configs
│ └── app.json
└── terraform # Terraform configs
├── modules # Terraform modules
│ ├── network
│ │ └── main.tf
│ └── service1
│ └── main.tf
├── main.tf
└── terraform.tfvars
~/infra-repo/
├── packer # Packer configs
│ └── app.json
└── terraform # Terraform configs
├── modules # Terraform modules
│ ├── network
│ │ └── main.tf
│ └── service1
│ └── main.tf
└── environments
├── non-prod
│ └── us-east-1
│ ├── main.tf
│ └── terraform.tfvars
└── prod
├── eu-west-1
│ ├── main.tf
│ └── terraform.tfvars
└── us-east-1
├── main.tf
└── terraform.tfvars

12. example1/main.tf
resource "random_pet" "bucket" {}
resource "aws_s3_bucket" "app" {
bucket = "fullstackfest-${ random_pet .bucket. id}"
acl = "public-read"
website {
index_document = "index.html"
}
}
data "template_file" "index" {
template = "${file("../../web/index.html")}"
vars {
BUILD_DETAILS = "${aws_s3_bucket .app.website_endpoint }"
}
}
resource "aws_s3_bucket_object" "object" {
bucket = "${aws_s3_bucket .app.id}"
key = "index.html"
content = "${data. template_file .index.rendered }"
etag = "${md5(data. template_file .index.rendered )}"
content_type = "text/html"
acl = "public-read"
}
output "app_website_endpoint" {
value = "${aws_s3_bucket .app.website_endpoint }"
}
FullStackFest!
${BUILD_DETAILS}
$ terraform init
...
$ terraform plan
...
$ terraform apply
...
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
Outputs:
app_website_endpoint =
fullstackfest-feasible-basilisk.s3-website-eu-west-1.amazonaws.c
om

13. example2/main.tf
variable "subnet_id" {
description = "ID of subnet where resources will be created"
}
variable "security_groups" {
description = "ID of security group EC2 instance will use"
}
variable "instance_type" {
description = "Type of EC2 instance to launch"
}
data "aws_ami" "app" {
most_recent = true
filter {
name = "name"
values = ["fullstackfest-demo-*" ]
}
}
resource "aws_instance" "app" {
ami = "${data. aws_ami.app.id}"
instance_type = "${var.instance_type }"
subnet_id = "${var.subnet_id }"
vpc_security_group_ids = ["${var.security_groups }"]
}
output "app_public_ip" {
description = "Public IP of EC2 instance running an
application"
value = "${aws_instance .app.public_ip }"
}
packer/app.json
{
"builders" : [
{
"ami_name" : "fullstackfest-demo-{{uuid | clean_ami_name}}" ,
"ami_description" : "FullStackFest demo AMI based on Amazon
Linux",
"instance_type" : "t2.micro" ,
"region" : "eu-west-1" ,
"type": "amazon-ebs" ,
"ssh_username" : "ec2-user" ,
"source_ami_filter" : {
"filters" : {
"virtualization-type" : "hvm",
"name": "amzn-ami-hvm-*-x86_64-gp2" ,
"root-device-type" : "ebs"
},
"owners" : [
"137112412989"
],
"most_recent" : true
}
}
],
"provisioners" : [
{
"type": "shell",
"inline" : [
"# Install nginx, copy index.html into web-root"
]
}
]
}

14. # Avoid hard-coded values in *.tf files, use data sources or
*.tfvars
data "aws_ami" "app" {
most_recent = true
filter {
name = "name"
values = ["fullstackfest-demo-*" ]
}
}
# Tag and name resources consistently
resource "aws_instance" "app" {
ami = "${data. aws_ami.app.id}"
instance_type = "${var.instance_type }"
subnet_id = "${var.subnet_id }"
vpc_security_group_ids = ["${var.security_groups }"]
tags {
Name = "fullstackfest-demo-${var. environment }"
}
}
variable "environment" {
description = "Name of environment to create infrastructure (eg,
staging, production)"
}
# terraform.tfvars
environment = "non-prod"
FTP (Frequent Terraform Problems)
● Avoid hard-coded values => use data
sources
● Tag and name resources consistently
Next: Terraform modules = reusability

15. module "sg_web" {
source = "git@github.com:terraform-community-modules/tf_aws_sg.git//sg_web?ref=v0.2.3"
security_group_name = "fullstackfest-demo-web"
vpc_id = "vpc-12345678"
source_cidr_block = ["0.0.0.0/0" ]
}
resource "aws_instance" "app" {
# ...
vpc_security_group_ids = ["${module. sg_web.security_group_id_web }"]
# ...
}
Terraform modules
● Versioning
● Public/private access
● Local dir or hosted
● Allows:
○ code reuse
○ encapsulate groups of resources
○ testing

16. Demo - infrastructure as code and deployment pipeline
https://github.com/antonbabenko/terraform-deployment-pipeline-talk

17. Further thoughts…
● Use linters (tflint), coding styles (fmt), pre-commit hooks
● Automate (no excuses)
● Terraform workspaces, Terragrunt, Atlantis by Hootsuite (super!)
● Version & release infrastructure as the app code
● Using pipelines to manage environments with infrastructure as code by Kief
Morris

18. Thank you!