Immutable infrastructure is a way to success, but what about the lifecycle of individual resources. This talk is about evolution of resources, code structure, Terraform coding tricks, composition and refactoring.

1. Lifecycle of a resource in Terraform
by Anton Babenko

2. Anton Babenko
Terraform AWS fanatic
Organiser of {HashiСorp, AWS, DevOps}
User Groups in Norway
DevOpsDays Oslo (29-30th October 2018)
github.com/antonbabenko
twitter.com/antonbabenko
linkedin.com/in/antonbabenko

5. Write, plan, and create infrastructure as code
www.terraform.io

7. Once created, infrastructure is going to
be updated…

8. And new versions of Terraform
will come out! Yay!!!

9. This talk is about evolution of resources
Code structure, Terraform coding tricks, refactoring

10. Terraform primitives
• Resources
• Data sources
• Variables
• Terraform state

11. Resources
• Create, Read, Update, Delete
• Lifecycles:
• ignore_changes
• prevent_destroy
• create_before_destroy

12. Data sources — read-only

13. Variables
• string, integer, boolean
• list
• map

14. Types of variables
Type of variable =>
string, integer,
boolean
list [] map {}
Command line Yes Yes Yes
*.tfvars Yes Yes Yes
Inside computing values (count,
lifecycle)
Yes No No
Inside other variables (string) Yes Yes Yes
Inside other variables (list) Yes Yes Yes, partially
Inside other variables (map) Yes Yes Yes

15. Terraform state
JSON file (*.tfstate) with information
about created resources
Humans should not touch it (often)

16. AWS S3 bucket

17. AWS EC2 Security Group

18. AWS EC2 Security Group module

19. Small infrastructure
As infrastructure grows and you manage more resources — how to group
them?

20. Resources + Data Sources = Module

21. Create Your First Module
https://www.terraform.io/docs/enterprise/guides/recommended-practices/part3.2.html#3-create-your-first-module

22. Types of Terraform modules
• Resource modules — very flexible, no relations to other modules, born to be
open-sourced
• Infrastructure modules — group of versioned resource modules, data-
sources, company-wide standards, code-generators (eg, jsonnet)

23. Usage of resource modules
Q: Why use resource modules
instead of resources?
A: Resources can’t be versioned,
but modules can.

24. Usage of infrastructure module

25. Modules tip #0
Check Terraform Registry before starting new resource module

26. Modules tip #1 — count types
Value of 'count' cannot be computed (issue #10857)

27. Modules tip #2 — scope
Remember the scope — no computed values in counts, no loops, no strict
assumptions on region/service availability.

28. Modules tip #3 — implementation
«Terraform module which creates RDS instance»
https://github.com/terraform-aws-modules/terraform-aws-rds

29. Modules tip #3 — implementation (example)

30. Modules tip #3 — usage (example)

31. Modules tip #4 — size
Usually infrastructure modules repositories have 99.9% waste — «terraform init» is slow

32. How to call modules?
There are two extremes:
1. Call many modules in one place
2. Call one module in one place

33. Composite pattern — many-in-one
Good:
1. Declare variables and outputs in fewer places
Bad:
1. Large blast radius — easier to break things
2. Locks everything at once
3. Single run vs orchestration concern (eg, first
run: data{0}=>resources{1}=>outputs{1}; second
run: data{0,1}=>resources{2}=>outputs{2})
4. No way to specify dependencies between
modules (depends_on)

34. Composite pattern — one-in-one
Good:
1. Small blast radius — harder to break
things
2. Possible to orchestrate, or chain runs
3. Easy to navigate
Bad:
1. Declare variables and outputs in
more places

35. Composite pattern — everything-in-between
The most popular choice

36. How to structure compositions?
1. Primary cloud provider services (VPC, ALB) or group of services (network, DB, shared)
2. Code changing frequency
3. Code change initiator (human or CI server)
4. Relation between components (eg, security group together with EC2 instance)
5. Used technology (AWS CodeDeploy, K8S, OpenShift)
6. Logical name of environment (staging, production)
7. Project

37. Code structure guidelines
• Try to keep Terraform state small and secure
• Use Terragrunt to orchestrate your configurations and to reduce copy-paste
• Let users to operate with «easy» values and keep interpolation magic hidden
most of the time

38. Poor man orchestration inception

39. WIP — https://github.com/antonbabenko/terraform-best-practices
Read more

40. Refactoring using Terraform 0.11

41. Refactoring
Any change (add feature, fix bug, improve design, optimise resource usage)
to the code which brings codebase closer to the desired state.
• incremental
• small
• accept the ugliness
• «edit & prey» vs «cover & modify»

42. Add new features/resources
Often easy, but…

43. Refactoring — conditional
Use existing resource or create a new one

44. Refactoring — lists
If user2 is removed then user3 and user4 will be recreated — this is a
problem for stateful resources like AWS IAM access keys.

45. jsonnet — alternative to lists for stateful
resources (eg, AWS IAM Access Keys)

46. Refactoring — import
• terraform import aws_iam_account_alias.this alias
• Use https://github.com/dtan4/terraforming to generate *.tf and tfstate from
existing AWS resources

47. Refactoring — rename/move

48. Refactoring — testing
•Basics — pre-commit (fmt, validate)
•Medium — review terraform plan
•On PR — Atlantis (runatlantis.io)
•Integration testing — terratest, awsspec

49. Refactoring — edge cases
• Test in different AWS regions (S3 signature, EC2 ClassicLink, IPv6)
• Check or open new github issues

50. Summary
• Terraform 0.11 has certain limitations — plan in advance!
• Use composition pattern — write less and simpler
• Reuse existing code and modules, fallback to documentation

51. Related Terraform projects
• https://github.com/antonbabenko/pre-commit-terraform — pre-commit git hooks to take care of
Terraform configurations (fmt, validate, terraform-docs)
• https://github.com/terraform-aws-modules/ — Collection of verified Terraform AWS modules
supported by the community
• https://github.com/antonbabenko/terraform-best-practices — Terraform best practices with
examples and arguments (WIP)
• https://cloudcraft.co/app?beta — «Export your AWS diagram as Terraform code» (tweet, modules.tf)
• https://github.com/antonbabenko/terrapin — Terraform module generator (POC)
• https://github.com/antonbabenko/terrible — Orchestrate Terraform configuration using Ansible
(POC)

52. Thank you!
Questions?
Code: github.com/antonbabenko
DM are open for all: twitter.com/antonbabenko