Infrastructure as Code with Terraform

Pedro J. Molina
Pedro J. MolinaFounder at Metadev en Metadev
Infrastructure as Code
with
2022.11.11
Dr. Pedro J. Molina
@pmolinam
Agenda
▪ Infrastructure as Code
▪ Immutable Infrastructure
▪ Cloud Providers and AWS
▪ Terraform
▪ Installation & Software Prerequisites
▪ Resources & Dependencies
▪ Execution Plans
▪ Industrial examples
▪ Best practices
▪ Exercises on AWS + Terraform
Get the material
1. Go to: https://github.com/metadevpro/terraform-aws-training
2. Clone the code examples:
git clone git@github.com:metadevpro/terraform-aws-training.git
3. Get credentials for an AWS account
Infrastructure as Code
Engineering Practice to define Infrastructure as code and configuration.
Main Properties:
▪ Repeatable
▪ Can be Versioned (with standard source code tools like git or hg)
▪ Robust
▪ Can be Automated
Immutable Infrastructure
Traditional Approach: PatchingServers
▪ Few items
▪ Named as pets
▪ Manual patching
▪ State unknown over time
▪ Improved by Ansible or Chef for automation
New Approachon scale: Immutable Infrastructure
▪ No patching. Managed as bacteria
▪ Destroy and recreate
▪ Well know-state
▪ Apply all security patches for better safety
Cloud Providers
Main Players
Amazon Web Services
Microsoft Azure
Google Cloud
Amazon Web Services
The first provider: inventors of the cloud (EC2, S3)
Leading innovationon cloud: AWS Lambda,Fargate, etc.
Very complete offeringof services.
Many Data-Centersaround the world.
Price competitive. Leaders and growingyear by year.
Terraform
https://www.terraform.io
Leading tool for manage Infrastructure as Code.
▪ Open Source
▪ Created by Hashi Corporation https://www.hashicorp.com
▪ Custom language to define infrastructure: HCL
Installation & Prereqs
Download & Install:
▪ Terraform from: https://www.terraform.io/downloads.html
▪ Copy local & include it in PATH
▪ AWS-CLI: https://aws.amazon.com/en/cli
▪ Visual Studio Code (editor) https://code.visualstudio.com
▪ Install Extension for Terraform
▪ Bash Shell (git shell, Cmder, or Conemu in Windows)
▪ PuTTY (ssh client for Windows) https://www.putty.org
Installation Cross-check
$ terraform -version
Terraform v0.14.7
$ aws --version
aws-cli/1.16.193 Python/3.6.0 Windows/10 botocore/1.12.183
Hashi Configuration Language (HCL)
Terraform uses *.tf files.
Simple Configuration DSL to describeResources and Desired State.
Similar to JSON syntax, but rich in expressiveness.
Samples:
resource "aws_instance" "web" {
ami = "ami-a1b2c3d4"
instance_type = "t2.micro"
}
resource "heroku_app" "app1" {
name = "my-cool-app"
region = "us"
config_vars = {
FOOBAR = "baz"
}
buildpacks = [
"heroku/go", "heroku/node"
]
}
Terraform: Resources
A Resource represent aconcrete (vendor-specific) Cloud Service we can
manipulate.
Resources has a well-knowtype with properties we must configure.
Resources are exposed and managed byProviders.
Examples:
aws-instance Represents a machine in AWS EC2 Service.
azurerm_virtual_machine Represents a virtual machine in Azure.
google_compute_instance Represents a virtual machine in Google Cloud.
Terraform: Providers
A Provider is a driver implementing the communication and automation
for an specific Cloud Provider.
Each provider expose more or less Resource types dependingon the offeringof
the CloudVendor, and the supportof the current Provider version.
Examples: Google,Github or Digital Ocean
See list of providers here: https://registry.terraform.io/browse/providers
Terraform: Configure Provider
provider.tf
# Configure the AWS Provider
provider "aws" {
region = "eu-west-3" # Paris
}
$ terraform init
Terraform: Input Variables
variables.tf
variable "author" {
description = "Operator’s name. Used as prefix."
type = string
default = "jessica"
}
$ terraform apply -var author=alice
Types:
▪ string
▪ number
▪ bool
▪ list
▪ map
▪ null
Terraform: Variables Interpolation
Variables can beinterpolated
Name = "${var.author}_machine1"
https://www.terraform.io/docs/configuration/expressions.html
https://www.terraform.io/docs/configuration/functions.html
Terraform: Output Variables
output.tf
output "instance_public_ip" {
value = aws_instance.machine01.public_ip
}
Sample one
Exercise 01
Create afirst Virtual Machine
▪ Setup credentials access to AWS
▪ Deploy on AWS in Paris Data Center
▪ Prefix with your name to avoid collisions
▪ Retrieve output public IP
▪ Use SSH Key to connect to the machine
$ ssh –i paris-keys.pem ec2-user@<ip>
ec-instance security-group
Terraform: Dependences
▪ Resources has dependences
▪ Forming a directed graph of resources
▪ Provision should follow a given order
▪ Deprovisining the reverse order
ec-instance
public-ip
esb-storage
vpc
dns
security group
load-balancing-group
rds-aurora-db
$ terraform graph http://www.webgraphviz.com/
Terraform: Desired State
Desired State: The ideal state described by the configuration (immutable).
Current State: The actual state in the infrastructure. Changes over time.
Services can be down. Provisioning can fail or lack or permissions.
Differences: The plan to add/remove/changes resources to achieve the
Desired State based in the Current State.
Terraform: State Management
Terraform uses:
▪ terraform.tfstate file to store last state know of a given infrastructure and
▪ terraform.tfstate.backup file to store the previous version.
There is service provide by Terraform athttps://app.terraform.io
to store the state in a shared central repository to be shared in a team.
For example: to prevent two provisionoperations at the same time.
Terraform: Basic Commands
terraforminit
terraformfmt
terraformvalidate
terraformplan
terraformapply
terraformdestroy
Terraform: Execution Plans
Sample:
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_instance.machine01 will be created
+ resource "aws_instance" "machine01" {
+ ami = "ami-007fae589fdf6e955"
+ arn = (known after apply)
+ associate_public_ip_address = true
+ get_password_data = false
+ instance_type = "t2.micro"
+ ipv6_addresses = (known after apply)
+ key_name = "paris-keys"
+ security_groups = (known after apply)
+ source_dest_check = true
+ subnet_id = (known after apply)
…
Create a static Web-site
Exercise 02
Create astatic web-site withS3
▪ Create a public bucket
▪ Upload html files and make it public
▪ Use the provided URL to access the web-site
S3-bucket iam-policy
Remote provisioners
Exercise 03
Provision aMachine
▪ Apply software updates: sudo yum update -y
▪ Install Docker
▪ Launch a container for a web app
aws-instance security-group
provision 1
provision 2
Terraform: Modules
Modules
▪ allows to create reusable
assets to be share between
projects
▪ Hides complexity(VPC creation
example)
▪ Registry for publicModules
https://registry.terraform.io/modules
/terraform-aws-
modules/vpc/aws/2.21.0
module "vpc" {
source = "git@github.com:terraform-
aws-modules/terraform-aws-vpc.git"
name = "${var.vpc_name}"
cidr = "172.29.208.0/20"
private_subnets = [
"172.29.208.0/24",
"172.29.209.0/24",
"172.29.210.0/24" ]
enable_nat_gateway = true
}
Terraform: Industrial Examples
Samples
1. E2E Tests scenarios for an Online University using Azure
in Spain
2. Dev/Staging/Prod environments for a mobile fintech app
in UK using AWS
3. Setup a private CI server in the cloud with Teamcity
Example
SQL Server
DBS
DB0 Security
AuditLog
MasterData
Environment QA
$ terraform apply
$ terraform destroy
Immutable Infrastructure
AWS
VPC 10.10.0.0/16
Subnet no-internet
10.10.51.0/24
Subnet db
10.10.21.0/24
Subnet private
10.10.1.0/24
Subnet public
10.10.11.0/24
Avaliability Zone 1 eu-west-2a Avaliability Zone 2 eu-west-2b
Router VPN Gateway
Customer
Gateway
VPN
Connection
Subnet no-internet
10.10.52.0/24
Subnet db
10.10.22.0/24
Subnet private
10.10.2.0/24
Subnet public
10.10.12.0/24
db
rabbitmq
services
nginx
services
db
rabbitmq
nginx
batch batch
3rd-party
Avaliability Zone 3 eu-west-2c
Subnet no-internet
10.10.53.0/24
Subnet db
10.10.23.0/24
Subnet private
10.10.3.0/24
Subnet public
10.10.13.0/24
services
db
rabbitmq
nginx
batch
Private CI Server
Exercise 04
Provision aPrivateTeamcityforContinuous Integration
▪ On the Cloud
▪ Usable for free for private projects till 100 projects
aws-instance
docker-compose
teamcity
security-group
Best Practices
▪Build your Terraform Scripts incrementally
▪Test them frequently
▪Encapsulate repeated blocks as modules
▪Incorporate existing infrastructure with terraformimport
▪Use variables to parametrize regions, AMIs, environment
prefix, etc.
▪Do notstore sensible credentials in repositories (inject later
as ENV vars)
▪Use provisioners (non declarative) as a last resort (prefer
packed images AMI) See Packer https://packer.io
Alternatives
Pulumi
https://www.pulumi.com
Infrastructure as Code. Imperative(uses JS), not declarative.
Compatible with (reuse) Terraformprovisioners.
AWSCloud Formation
https://aws.amazon.com/es/cloudformation
Provides templates(JSON/YAML based) to create resourcesin AWS. AWS only.
Azure Resource Manager
https://docs.microsoft.com/es-es/azure/azure-resource-manager/templates/overview
Similartemplate approach to Cloud Formation for Azure only (JSON based).
https://metadev.pro
@metad3v
1 de 34

Más contenido relacionado

Similar a Infrastructure as Code with Terraform(20)

Infrastructure as Code for Azure: ARM or Terraform?Infrastructure as Code for Azure: ARM or Terraform?
Infrastructure as Code for Azure: ARM or Terraform?
Katherine Golovinova3.8K vistas
Building the TribefireOperatorBuilding the TribefireOperator
Building the TribefireOperator
Oliver Moser237 vistas
HotLink DR ExpressHotLink DR Express
HotLink DR Express
dean16091K vistas
Infrastructure as Code with TerraformInfrastructure as Code with Terraform
Infrastructure as Code with Terraform
Mathieu Herbert409 vistas
Docker Security workshop slidesDocker Security workshop slides
Docker Security workshop slides
Docker, Inc.5.3K vistas
Managing Infrastructure as CodeManaging Infrastructure as Code
Managing Infrastructure as Code
Allan Shone691 vistas
Chapter-11.pdfChapter-11.pdf
Chapter-11.pdf
AlphonsePja2 vistas
DevOps Enabling Your TeamDevOps Enabling Your Team
DevOps Enabling Your Team
GR8Conf395 vistas
Multi Tenancy With Python and DjangoMulti Tenancy With Python and Django
Multi Tenancy With Python and Django
scottcrespo11.6K vistas

Más de Pedro J. Molina(20)

TerraformTerraform
Terraform
Pedro J. Molina5 vistas
Are Startups for me?Are Startups for me?
Are Startups for me?
Pedro J. Molina465 vistas
Meow DemoMeow Demo
Meow Demo
Pedro J. Molina228 vistas
Essential as the base for Web DSLsEssential as the base for Web DSLs
Essential as the base for Web DSLs
Pedro J. Molina300 vistas
Esencia de Web ComponentsEsencia de Web Components
Esencia de Web Components
Pedro J. Molina313 vistas
Esencia de web componentsEsencia de web components
Esencia de web components
Pedro J. Molina323 vistas
OpenAPI 3.0.2OpenAPI 3.0.2
OpenAPI 3.0.2
Pedro J. Molina1.3K vistas
QuidQuid
Quid
Pedro J. Molina430 vistas
Securizando por construcción mediante MDESecurizando por construcción mediante MDE
Securizando por construcción mediante MDE
Pedro J. Molina237 vistas
Building APIs with the OpenApi SpecBuilding APIs with the OpenApi Spec
Building APIs with the OpenApi Spec
Pedro J. Molina1K vistas
Micro vs Nano (servicios)Micro vs Nano (servicios)
Micro vs Nano (servicios)
Pedro J. Molina2K vistas
Diseño de APIs con OpenAPIDiseño de APIs con OpenAPI
Diseño de APIs con OpenAPI
Pedro J. Molina2.3K vistas
SVQDC 2017 Tecnologías para MicroserviciosSVQDC 2017 Tecnologías para Microservicios
SVQDC 2017 Tecnologías para Microservicios
Pedro J. Molina1.1K vistas
Introducción a AngularIntroducción a Angular
Introducción a Angular
Pedro J. Molina3.2K vistas
Tecnologías para microserviciosTecnologías para microservicios
Tecnologías para microservicios
Pedro J. Molina3.6K vistas
Microservicios sobre MEAN StackMicroservicios sobre MEAN Stack
Microservicios sobre MEAN Stack
Pedro J. Molina1.7K vistas
Hivepod: Casos de uso en OpenDataHivepod: Casos de uso en OpenData
Hivepod: Casos de uso en OpenData
Pedro J. Molina797 vistas

Infrastructure as Code with Terraform

  • 2. Dr. Pedro J. Molina @pmolinam
  • 3. Agenda ▪ Infrastructure as Code ▪ Immutable Infrastructure ▪ Cloud Providers and AWS ▪ Terraform ▪ Installation & Software Prerequisites ▪ Resources & Dependencies ▪ Execution Plans ▪ Industrial examples ▪ Best practices ▪ Exercises on AWS + Terraform
  • 4. Get the material 1. Go to: https://github.com/metadevpro/terraform-aws-training 2. Clone the code examples: git clone git@github.com:metadevpro/terraform-aws-training.git 3. Get credentials for an AWS account
  • 5. Infrastructure as Code Engineering Practice to define Infrastructure as code and configuration. Main Properties: ▪ Repeatable ▪ Can be Versioned (with standard source code tools like git or hg) ▪ Robust ▪ Can be Automated
  • 6. Immutable Infrastructure Traditional Approach: PatchingServers ▪ Few items ▪ Named as pets ▪ Manual patching ▪ State unknown over time ▪ Improved by Ansible or Chef for automation New Approachon scale: Immutable Infrastructure ▪ No patching. Managed as bacteria ▪ Destroy and recreate ▪ Well know-state ▪ Apply all security patches for better safety
  • 7. Cloud Providers Main Players Amazon Web Services Microsoft Azure Google Cloud
  • 8. Amazon Web Services The first provider: inventors of the cloud (EC2, S3) Leading innovationon cloud: AWS Lambda,Fargate, etc. Very complete offeringof services. Many Data-Centersaround the world. Price competitive. Leaders and growingyear by year.
  • 9. Terraform https://www.terraform.io Leading tool for manage Infrastructure as Code. ▪ Open Source ▪ Created by Hashi Corporation https://www.hashicorp.com ▪ Custom language to define infrastructure: HCL
  • 10. Installation & Prereqs Download & Install: ▪ Terraform from: https://www.terraform.io/downloads.html ▪ Copy local & include it in PATH ▪ AWS-CLI: https://aws.amazon.com/en/cli ▪ Visual Studio Code (editor) https://code.visualstudio.com ▪ Install Extension for Terraform ▪ Bash Shell (git shell, Cmder, or Conemu in Windows) ▪ PuTTY (ssh client for Windows) https://www.putty.org
  • 11. Installation Cross-check $ terraform -version Terraform v0.14.7 $ aws --version aws-cli/1.16.193 Python/3.6.0 Windows/10 botocore/1.12.183
  • 12. Hashi Configuration Language (HCL) Terraform uses *.tf files. Simple Configuration DSL to describeResources and Desired State. Similar to JSON syntax, but rich in expressiveness. Samples: resource "aws_instance" "web" { ami = "ami-a1b2c3d4" instance_type = "t2.micro" } resource "heroku_app" "app1" { name = "my-cool-app" region = "us" config_vars = { FOOBAR = "baz" } buildpacks = [ "heroku/go", "heroku/node" ] }
  • 13. Terraform: Resources A Resource represent aconcrete (vendor-specific) Cloud Service we can manipulate. Resources has a well-knowtype with properties we must configure. Resources are exposed and managed byProviders. Examples: aws-instance Represents a machine in AWS EC2 Service. azurerm_virtual_machine Represents a virtual machine in Azure. google_compute_instance Represents a virtual machine in Google Cloud.
  • 14. Terraform: Providers A Provider is a driver implementing the communication and automation for an specific Cloud Provider. Each provider expose more or less Resource types dependingon the offeringof the CloudVendor, and the supportof the current Provider version. Examples: Google,Github or Digital Ocean See list of providers here: https://registry.terraform.io/browse/providers
  • 15. Terraform: Configure Provider provider.tf # Configure the AWS Provider provider "aws" { region = "eu-west-3" # Paris } $ terraform init
  • 16. Terraform: Input Variables variables.tf variable "author" { description = "Operator’s name. Used as prefix." type = string default = "jessica" } $ terraform apply -var author=alice Types: ▪ string ▪ number ▪ bool ▪ list ▪ map ▪ null
  • 17. Terraform: Variables Interpolation Variables can beinterpolated Name = "${var.author}_machine1" https://www.terraform.io/docs/configuration/expressions.html https://www.terraform.io/docs/configuration/functions.html
  • 18. Terraform: Output Variables output.tf output "instance_public_ip" { value = aws_instance.machine01.public_ip }
  • 19. Sample one Exercise 01 Create afirst Virtual Machine ▪ Setup credentials access to AWS ▪ Deploy on AWS in Paris Data Center ▪ Prefix with your name to avoid collisions ▪ Retrieve output public IP ▪ Use SSH Key to connect to the machine $ ssh –i paris-keys.pem ec2-user@<ip> ec-instance security-group
  • 20. Terraform: Dependences ▪ Resources has dependences ▪ Forming a directed graph of resources ▪ Provision should follow a given order ▪ Deprovisining the reverse order ec-instance public-ip esb-storage vpc dns security group load-balancing-group rds-aurora-db $ terraform graph http://www.webgraphviz.com/
  • 21. Terraform: Desired State Desired State: The ideal state described by the configuration (immutable). Current State: The actual state in the infrastructure. Changes over time. Services can be down. Provisioning can fail or lack or permissions. Differences: The plan to add/remove/changes resources to achieve the Desired State based in the Current State.
  • 22. Terraform: State Management Terraform uses: ▪ terraform.tfstate file to store last state know of a given infrastructure and ▪ terraform.tfstate.backup file to store the previous version. There is service provide by Terraform athttps://app.terraform.io to store the state in a shared central repository to be shared in a team. For example: to prevent two provisionoperations at the same time.
  • 24. Terraform: Execution Plans Sample: An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # aws_instance.machine01 will be created + resource "aws_instance" "machine01" { + ami = "ami-007fae589fdf6e955" + arn = (known after apply) + associate_public_ip_address = true + get_password_data = false + instance_type = "t2.micro" + ipv6_addresses = (known after apply) + key_name = "paris-keys" + security_groups = (known after apply) + source_dest_check = true + subnet_id = (known after apply) …
  • 25. Create a static Web-site Exercise 02 Create astatic web-site withS3 ▪ Create a public bucket ▪ Upload html files and make it public ▪ Use the provided URL to access the web-site S3-bucket iam-policy
  • 26. Remote provisioners Exercise 03 Provision aMachine ▪ Apply software updates: sudo yum update -y ▪ Install Docker ▪ Launch a container for a web app aws-instance security-group provision 1 provision 2
  • 27. Terraform: Modules Modules ▪ allows to create reusable assets to be share between projects ▪ Hides complexity(VPC creation example) ▪ Registry for publicModules https://registry.terraform.io/modules /terraform-aws- modules/vpc/aws/2.21.0 module "vpc" { source = "git@github.com:terraform- aws-modules/terraform-aws-vpc.git" name = "${var.vpc_name}" cidr = "172.29.208.0/20" private_subnets = [ "172.29.208.0/24", "172.29.209.0/24", "172.29.210.0/24" ] enable_nat_gateway = true }
  • 28. Terraform: Industrial Examples Samples 1. E2E Tests scenarios for an Online University using Azure in Spain 2. Dev/Staging/Prod environments for a mobile fintech app in UK using AWS 3. Setup a private CI server in the cloud with Teamcity
  • 30. Immutable Infrastructure AWS VPC 10.10.0.0/16 Subnet no-internet 10.10.51.0/24 Subnet db 10.10.21.0/24 Subnet private 10.10.1.0/24 Subnet public 10.10.11.0/24 Avaliability Zone 1 eu-west-2a Avaliability Zone 2 eu-west-2b Router VPN Gateway Customer Gateway VPN Connection Subnet no-internet 10.10.52.0/24 Subnet db 10.10.22.0/24 Subnet private 10.10.2.0/24 Subnet public 10.10.12.0/24 db rabbitmq services nginx services db rabbitmq nginx batch batch 3rd-party Avaliability Zone 3 eu-west-2c Subnet no-internet 10.10.53.0/24 Subnet db 10.10.23.0/24 Subnet private 10.10.3.0/24 Subnet public 10.10.13.0/24 services db rabbitmq nginx batch
  • 31. Private CI Server Exercise 04 Provision aPrivateTeamcityforContinuous Integration ▪ On the Cloud ▪ Usable for free for private projects till 100 projects aws-instance docker-compose teamcity security-group
  • 32. Best Practices ▪Build your Terraform Scripts incrementally ▪Test them frequently ▪Encapsulate repeated blocks as modules ▪Incorporate existing infrastructure with terraformimport ▪Use variables to parametrize regions, AMIs, environment prefix, etc. ▪Do notstore sensible credentials in repositories (inject later as ENV vars) ▪Use provisioners (non declarative) as a last resort (prefer packed images AMI) See Packer https://packer.io
  • 33. Alternatives Pulumi https://www.pulumi.com Infrastructure as Code. Imperative(uses JS), not declarative. Compatible with (reuse) Terraformprovisioners. AWSCloud Formation https://aws.amazon.com/es/cloudformation Provides templates(JSON/YAML based) to create resourcesin AWS. AWS only. Azure Resource Manager https://docs.microsoft.com/es-es/azure/azure-resource-manager/templates/overview Similartemplate approach to Cloud Formation for Azure only (JSON based).