DevOps | AWS Loft Architecture Week | Wednesday, August 17

1. © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
DevOps Week 2016
AWS CloudFormation Best
Practices
August 2016
Hisham Baz, Solutions Architect

2. Infrastructure as code
• Scalability (anything manual is not scalable)
• Reliability
• Reproduction/Duplication
• Environment consistency
• Auditability/Record Keeping
• Security
• Governance

3. OpsWorks CloudFormationElastic Beanstalk
DevOps framework for
application lifecycle
management and
automation
Templates to deploy &
update infrastructure as
code
Automated resource
management – web
apps made easy
DIY /
On Demand
DIY, on demand
resources: EC2, S3,
custom AMI’s, etc.
Control
Deployment and management options
Convenience Control

4. AWS CloudFormation
• Create templates of the infrastructure
• CloudFormation provisions AWS resources in
order
• Version control/replicate/update with
infrastructure-as-code
• Integrates with development, CI/CD,
management tools

5. Application stack example
Template File
Defining Stack
Git
Subversion
Mercurial
Dev
Test
Prod
The entire application can be
represented in an AWS
CloudFormation template.
Use the version
control system of
your choice to store
and track changes to
this template
Build out multiple
environments, such
as for Development,
Test, and Production
using the template

6. Template Anatomy
{
"Description" : "Create an EC2 instance.”,
"Resources" : {
"Ec2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"KeyName" : “my-key-pair”,
"ImageId" : "ami-75g0061f”,
“InstanceType” : “m1.medium”
}
}
}
}

7. Editing Templates Best Practices

8. Stub templates with the designer

9. Reverse engineer with CloudFormer

10. Use change management tools
• Store templates in version control
• Automate deployment using CICD
• Check templates using unit tests
• Run templates, validates outputs, then tear
down

11. Avoid manual resource modifications
• Avoid making quick-fixes out of band
• Update your stacks with CloudFormation
• Do not manually change resources
• Consider using resource based permissions to
limit ability to make changes directly

12. Preview updates with Change Sets

13. Managing costs with budgets

14. Learn the intrinsic functions

15. Fn::FindInMap
"Mappings" : {
"RegionMap" : {
"us-east-1" : { "32" : "ami-6411e20d", "64" : "ami-
7a11e213" },
"us-west-1" : { "32" : "ami-c9c7978c", "64" : "ami-
cfc7978a" },
"eu-west-1" : { "32" : "ami-37c2f643", "64" : "ami-
31c2f645" },
"ap-southeast-1" : { "32" : "ami-66f28c34", "64" : "ami-
60f28c32" },
"ap-northeast-1" : { "32" : "ami-9c03a89d", "64" : "ami-
a003a8a1" }
}
},

16. Fn::FindInMap
"Resources" : {
"myEC2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"ImageId" : { "Fn::FindInMap" : [
"RegionMap", { "Ref" : "AWS::Region" }, "32"]},
"InstanceType" : "m1.small"
}
}
}

17. Extending AWS CloudFormation

18. security group
Auto Scaling group
EC2
instance
Elastic Load
Balancing
ElastiCache
Memcached cluster
Software pkgs,
config, & dataCloudWatch
alarms
Web Analytics
Service
AWS
CloudFormation
Provision
AWS Resources
“Create, Update,
Rollback, or Delete”
Extend with stack events
Worker
Amazon
SNS Topic
Stack Events

19. security group
Auto Scaling group
EC2
instance
Elastic Load
Balancing
ElastiCache
Memcached cluster
Software pkgs,
config, & dataCloudWatch
alarms
Web Analytics
Service
AWS
CloudFormation
Provision
AWS Resources
Extend with custom resources
"Resources" : {
"WebAnalyticsTrackingID" : {
"Type" : "Custom::WebAnalyticsService::TrackingID",
"Properties" : {
"ServiceToken" : "arn:aws:sns:...",
"Target" : {"Fn::GetAtt" : ["LoadBalancer",
"DNSName"]},
"Plan" : "Gold"
}
},
...
“Success” + Metadata
“Create, Update, Rollback, or Delete”
+ Metadata

20. Lambda-backed custom resources
security group
Auto Scaling group
EC2
instance
Elastic Load
Balancing
ElastiCache
memcached cluster
Software pkgs,
config, & dataCloudWatch
alarms
Your AWS CloudFormation stack
// Implement custom logic here
Look up an AMI ID
Your AWS Lambda functions
Look up VPC ID and Subnet ID
Reverse an IP address
Lambda-powered
custom resources

21. Security Best Practices

22. Audit operations
Store/
Archive
Troubleshoot
Monitor & Alarm
You are
making API
calls...
On a growing
set of AWS
services around
the world..
CloudTrail is
continuously
recording
API calls

23. Publish templates with Service Catalog
• For larger organizations, limit user access to
CloudFormation directly
• Developers create standard templates
• Publish to Service Catalog for consumption

24. Restricting user access
• Only allow specific templates
{
"Effect":"Allow”,
"Action":[
"cloudformation:CreateStack",
"cloudformation:UpdateStack”
],
"Condition":{
"ForAllValues:StringLike":{
"cloudformation:TemplateUrl":
["https://.amazonaws.com/TestBucket/*"]
}
}

25. Restricting user access
• Only allow certain users to update
{
"Effect":"Allow”,
"Action":[
"cloudformation:UpdateStack”
],
"Condition":{
"ForAllValues:StringEquals":{
"cloudformation:StackPolicyUrl":
["https://.amazonaws.com/TestBucket/Foo.json"]
}
}
}

26. Restricting user access
• Only allow specific resource types
{
"Effect":"Allow”,
"Action":[
"cloudformation:CreateStack”
],
"Condition":{
"ForAllValues:StringEquals":{
"cloudformation:ResourceType":
[”AWS::EC2::Instance”…]
}
}
}

27. Restricting user access
• Deny specific resource types
{
"Effect":"Allow”,
"Action":[
"cloudformation:CreateStack”
]
},
{
"Effect":”Deny”,
"Action":[
"cloudformation:CreateStack”
]
"Condition":{
"ForAnyValue:StringLike":{
"cloudformation:ResourceType":
[”AWS::IAM::*"]

28. Limit resource types
• Programmatically restrict access to resource types
• CreateStack and UpdateStack take a new parameter
• Restrict the set of resources that can be created
• Independent of any user policies
$ aws cloudformation create-stack … --resource-types=“[AWS::EC2::*, AWS::RDS::DBInstance, Custom::MyCustomResource]”

29. Modularization Best Practices

30. Single responsibility principle
• Use nested stacks to break up large templates
• Limit one template to a single service
• Organize templates according to team structure

34. Re-using Templates across AWS Regions
• Consider environmental or regional differences
• Amazon EC2 image Ids
• VPC environment or “classic” environment
• Available instance types
• IAM policy principals
• Endpoint names
• Amazon Resource Names (arns)

35. Re-usable Templates – “Pseudo-Parameters”
• Use “pseudo-parameters” to retrieve environmental data
– Account Id
– Region
– Stack Name and Id
"LogsBucketPolicy": {
"Type": "AWS::S3::BucketPolicy",
"Properties": {
"Bucket": {"Ref": "LogsBucket”},
"PolicyDocument": {
"Version": "2008-10-17",
"Statement": [{
"Sid": "ELBAccessLogs",
"Effect": "Allow",
"Resource": {
"Fn::Join": [ "", [ “arn:aws:s3:::",
{ "Ref": "LogsBucket" }, "/", "Logs",
"/AWSLogs/", { "Ref": "AWS::AccountId" }, "/*” ]]
},
"Principal": …,
"Action": [ "s3:PutObject" ]
}]
}
}
},

36. Re-usable Templates - Using mappings
Use mappings to define variables
• Single place for configuration
• Re-usable within the template
"LogsBucketPolicy": {
"Type": "AWS::S3::BucketPolicy",
"Properties": {
"Bucket": {"Ref": "LogsBucket”},
"PolicyDocument": {
"Version": "2008-10-17",
"Statement": [{
"Sid": "ELBAccessLogs",
"Effect": "Allow",
"Resource": {
"Fn::Join": [ "", [
{ "Fn::FindInMap" : ["RegionalConfig",
{"Ref" : "AWS::Region"},
"ArnPrefix”]},
"s3:::”, { "Ref": "LogsBucket" }, "/",
"Logs",
"/AWSLogs/”,
{ "Ref": "AWS::AccountId" }, "/*" ] ]
},
:
“Mappings” : {
“RegionalConfig” : {
“us-east-1” : {
“AMI” :
“ami-12345678”,
”ELBAccountId":
"127311923021”,
“ArnPrefix” :
“arn:aws:”
},
“us-west-1” : {
“AMI” :
“ami-98765432”
”ELBAccountId":
“027434742980"
“ArnPrefix” :
“arn:aws:”
},
:
}
}

37. Re-usable Templates – Using conditionals
Use conditionals to customize
resources and parameters
"DBEC2SG": {
"Type": "AWS::EC2::SecurityGroup",
"Condition" : "Is-EC2-VPC",
"Properties" : {…}
},
"DBSG": {
"Type": "AWS::RDS::DBSecurityGroup",
"Condition" : "Is-EC2-Classic",
"Properties": {…}
},
"MySQLDatabase": {
"Type": "AWS::RDS::DBInstance",
"Properties": {
:
"VPCSecurityGroups": { "Fn::If" : [ "Is-EC2-VPC",
[ { "Fn::GetAtt": [ "DBEC2SG", "GroupId" ] } ],
{ "Ref" : "AWS::NoValue"}]},
"DBSecurityGroups": { "Fn::If" : [ "Is-EC2-Classic",
[ { "Ref": "DBSG" } ],
{ "Ref" : "AWS::NoValue"}]}
}
}
}
"Conditions" : {
"Is-EC2-VPC” : { "Fn::Or" : [
{"Fn::Equals" : [
{"Ref” : "AWS::Region"},
"eu-central-1" ]},
{"Fn::Equals" : [
{"Ref" : "AWS::Region"},
"cn-north-1" ]}]},
"Is-EC2-Classic" : { "Fn::Not" : [
{ "Condition" : "Is-EC2-VPC"}]}
},

38. Best Practices Summary
• Editing
– Stub templates with the designer
– Reverse engineer with
CloudFormer
– Use change management tools
– Avoid manual resource
modifications
– Preview updates with Change
Sets
– Manage costs with budgets
– Learn the intrinsic functions
• Extend
– Use stack events to trigger
external integration
– Create custom resources
for integrations
– Use Lambda custom
resources

39. Best Practices Summary
• Security
– Audit operations with
CloudTrail
– Publish with Service Catalog
– Restrict specific templates
– Limit resource types
• Modularization
– Single responsibility principle
– Plan for multi-region
– Use Pseudo-Parameters
– Use Mappings
– Use Conditionals

40. Questions?