AWS Summit 2014 Brisbane - Breakout 6 Technical deep dive in to 10 AWS Cloud best practices with in-depth look at the tips and tricks of architecting on the AWS platform. Presenter: Dean Samuels, Solutions Architect, Amazon Web Services

1. © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
Black Belt Tips on AWS
Dean Samuels,
Solutions Architect, Amazon Web Services

2. AWS Rapid Pace of Innovation!
2009!
Amazon RDS!
Amazon VPC!
Auto Scaling!
Elastic Load!
!
Balancing!
+4
8!
2010!
Amazon SNS!
AWS Identity !
!
& Access !
!
Management!
Amazon Route 53!
+6
1!
2011!
Amazon !
!
ElastiCache!
Amazon SES!
AWS !
!
CloudFormation!
AWS Direct !
!
Connect!
AWS Elastic !
!
Beanstalk!
GovCloud!
+82!
Amazon !
!
CloudTrail!
Amazon !
!
CloudHSM!
Amazon !
!
WorkSpaces!
Amazon Kinesis!
Amazon Elastic!
!
Transcoder!
Amazon !
!
AppStream!
AWS OpsWorks!
+280!
2013!
Amazon SWF!
Amazon Redshift!
Amazon Glacier!
Amazon !!
Dynamo DB!
Amazon !
!
CloudSearch!
AWS Storage!
!
Gateway!
AWS Data !
!
Pipeline!
+159!
2012!
Since inception AWS has:!
!
• Released 927 new services and features !
• Introduced over 35 major new services!
• Announced 45 price reductions!
!
!
2008!
+24!
Amazon EBS!
Amazon!
!
CloudFront!
+270!
2014!
Amazon Cognito!
Amazon Zocalo!
Amazon Mobile!
!
Analytics!
*as of July 31, 2014

3. Ninja Tips
• Compute and Networking
• Storage & Content Delivery
• Deployment & Management
• Security
• Big Data & App Services……maybe!

4. • Black Belt Tip
– Route53 & Elastic Load Balancing
• Cross-Zone Load Balancing
• Application Failover via DNSMeet Steve
Challenges
• Use of AWS is starting to grow
• Focus on end user experience
• Minimise blast radius in event of issues
• Prefers compartmentalization
• Hitting AWS account limits

5. • Route 53 DNS Failover
ELB & Route53
• Cross-Zone Load Balancing

6. Meet Steve
• Black Belt Tip
– Route53 & Elastic Load Balancing
• Cross-Zone Load Balancing
• Application Failover via DNS
• Ninja Tip
– VPC Peering
• Trust thy neighbour!
– VPC peering within an account
– VPC peering between accounts
Challenges
• Use of AWS is starting to grow
• Focus on end user experience
• Minimise blast radius in event of issues
• Prefers compartmentalization
• Hitting AWS account limits

7. VPC Peering
Steve’s Shared Services VPC
10.1.0.0/16
Steve’s Workspaces VPC
192.168.0.0/20
Steve’s Enterprise Apps VPC
172.16.0.0/16
Steve’s Web Apps VPC
10.11.0.0/16
Steve’s Proxy VPC
10.20.10.0/24
Internet
Dean’s WAF VPC
10.100.0.0/16
George’s Test/Dev VPC
10.10.0.0/16

8. • Black Belt Tip
– Storage Gateway File Shares
• S3 Backed NAS
– Large volume file shares, no upfront cost
– On-premise or in the AWS Cloud
This is Gwen
Challenges
• Leverages multiple storage tiers on AWS
• EBS for persistent block storage
• S3 for backups and serving web & media
• Glacier for archiving data
• But storage is starting to become costly…
even on AWS
• Favours the pay for what you use model
with S3 rather than what you provision
• Requires high performance block storage

9. Next Generation Storage
File Servers
Corporate Data center
AWS Cloud
Internet
or
WAN
SSL
On-Premise AWS
Storage Gateway
Cache & Upload Buffer Storage
Direct Attached or Storage Area
Network Disks
iSCSI
Cached-Volumes
Multi-Terabyte
AWS Storage
Gateway Service
“Block” Volumes
@ S3 Prices
“Block” Volumes
@ S3 Prices
Encrypted &
Compressed
Volume
Snapshots
EC2
File Servers
iSCSI
Cached-Volumes
Multi-Terabyte
CIFS/
NFS
Clients
CIFS/
NFS
EC2 Clients
Third-Party options too:
• Riverbed SteelStore
• SoftNAS
• Maginatics
EC2 AWS Cached
Storage Gateway
Cache &
Upload Buffer
EBS PIOPS

10. • Black Belt Tip
– Storage Gateway File Shares
• S3 Backed NAS
– Large volume file shares, no upfront cost
– On-premise or in the AWS Cloud
• Ninja Tip
– Instance Storage
• Normally ephemeral storage
– Using replication = durable storage
– EBS PIOPs, General Purpose SSDs
and Enhanced Networking
This is Gwen
Challenges
• Leverages multiple storage tiers on AWS
• EBS for persistent block storage
• S3 for backups and serving web & media
• Glacier for archiving data
• But storage is starting to become costly…
even on AWS
• Favours the pay for what you use model
with S3 rather than what you provision
• Requires high performance block storage

11. High Speed* & High Density*
Instance storage for durable data
Instance Storage with sync to EBS Instance Storage to Instance Storage to EBS
*I2 and C3 Instances:
- Multiple 10s & 100’s GB SSD-based instance storage
- Enhanced Networking = Higher PPS and lower jitter & latency
EBS Optimized
MDADM
RAID 0
array
DRBD
protocol A
(asynchronous)
Up to 50,000 IOPs = 800MBs
General Network
Traffic
EBS PIOPS or GP2
SSD Backed
Data Store
EC2 Instance
MDADM
RAID 0 or 1+0
array
HDD
or
SSD (100,000s
IOPS) Enhanced
Networking*

12. • Black Belt Tip
– AWS = Programmable Resources
• AWS Support is an API
• Use Resource Tags for management
• Centralised logging and notification
Say Hi to Felix
Challenges
• Still very manual deployment and
configuration processes of AWS resources
• Lots of human interaction
• Starting to get resource sprawl – harder to
manage
• Not everything is supported by
CloudFormation

13. Everything is an API
• Monitoring Your Service Limits
– Via Service API
• aws iam get-account-summary
• aws autoscaling describe-account-limits
• aws ec2 describe-account-attributes
• aws ses get-send-quota
– Via Trusted Advisor
• aws support describe-trusted-advisor-check-result --check-id eW7HH0l7J9
--language en
• Accessing Support via API
– Integrate with your own management/monitoring systems
– Automatically log tickets via CloudFormation

14. Resource Management with Tags
#!/usr/bin/ruby
require 'aws-sdk'
AWS.regions.sort_by(&:name).each do |region|
puts region.name
region.ec2.instances.each do |instance|
if instance.status == :stopped and instance.tags.to_h.has_key?('DevProjectA')
instance.start
puts "t#{instance.id} starting"
end
end
end
for region in $(aws ec2 describe-regions --query 'Regions[*].RegionName' --output text)
do
echo ${region}
aws ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId]' --filters
"Name=instance-state-name,Values=running" "Name=tag-key, Values=Uptime, Name=tag-value,
Values=BusinessHoursOnly" --output text --region ${region} | xargs aws ec2 stop-instances --
instance-ids --region ${region} 2> /dev/null
done
Ruby SDK
AWS CLI

15. Centralised Log Collection
• CloudTrail
– Get log files of API calls made on your AWS account
• CloudWatch Logs
– Store and Monitor OS & Application Log Files with Amazon CloudWatch
• Service Logs
– RDS, ELB, S3, CloudFront, EMR
• Detailed Billing Reports
– Cost Allocation For Customer Bills
All stored in S3

16. • Black Belt Tip
– AWS = Programmable Resources
• AWS Support is an API
• Use Resource Tags for management
• Centralised logging and notification
• Ninja Tip
– CloudFormation
• Taking it to the next level!
– Custom Resources
Say Hi to Felix
Challenges
• Still very manual deployment and
configuration processes of AWS resources
• Lots of human interaction
• Starting to get resource sprawl – harder to
manage
• Not everything is supported by
CloudFormation

17. CloudFormation Custom Resources
Region
SQS Queue
AWS
CloudFormation
Custom Resource
Topic
Auto scaling Group
Custom Resource
Implementation
• Add New Resources
– Including AWS resources not currently
supported by CFN
• Interact with the CloudFormation
Workflow
• Inject dynamic data into a stack
• Extend the capabilities of existing
resources
• Data management via
CloudFormation
• It’s really simple if you use
aws-cfn-resource-bridge
– Install or fork from
https://github.com/aws/aws-cfn-resource-bridge
Create
Parameter1:Value1
Parameter2:Value2
….
Parametern:Valuen
Data
Export
Data
Import
DynamoDB S3Datapipeline
1
2 3
4
5
6
Output
Parameter1:Value1
Parameter2:Value2
….
Parametern:Valuen

18. CloudFormation Custom Resources
Region
SQS Queue
AWS
CloudFormation
Custom Resource
Topic
Auto scaling Group
Custom Resource
Implementation
• Add New Resources
– Including AWS resources not currently
supported by CFN
• Interact with the CloudFormation
Workflow
• Inject dynamic data into a stack
• Extend the capabilities of existing
resources
• Data management via
CloudFormation
• It’s really simple if you use
aws-cfn-resource-bridge
– Install or fork from
https://github.com/aws/aws-cfn-resource-bridge
Delete
Parameter1:Value1
Parameter2:Value2
….
Parametern:Valuen
Data
Import
Data
Export
DynamoDB S3Datapipeline
1 2
3
4
5
Output
Parameter1:Value1
Parameter2:Value2
….
Parametern:Valuen
6

19. What’s up Alex?
• Black Belt Tip
– IAM Roles with EC2
• Don’t leave home without it!
Challenges
• Admin users with no MFA
• Users leaving credentials in software
• Users not rotating their credentials
• Users not using strong password
policies
• Finds it hard to keep track of
individual IAM identifies for users

20. IAM Roles for EC2 Instances
AWS Cloud
Amazon
S3
Amazon
DynamoDB
Your
Application
AWS IAM
Your
Application
Your
Application
Your
Application
Auto
Scaling
Your
Application
Auto
Scaling
Role: RW access to
objects, items and
instances
• Eliminates use of long-term credentials
• Automatic credential rotation
• Less coding – AWS SDK does all the work
• Easier and more Secure!
Amazon
EC2

21. What’s up Alex?
• Black Belt Tip
– IAM Roles with EC2
• Don’t leave home without it!
• Ninja Tip
– Limit number of IAM Users
• Use IAM Roles instead
– Cross-Account IAM Access
– Identity Federation
Challenges
• Admin users with no MFA
• Users leaving credentials in software
• Users not rotating their credentials
• Users not using strong password
policies
• Finds it hard to keep track of
individual IAM identifies for users

22. dsamuel@amazon.com
Acct ID: 111122223333
ec2-role
{
"Statement":
[
{
"Action":
[
"ec2:StartInstances",
"ec2:StopInstances"
],
"Effect":
"Allow",
"Resource":
"*"
}
]
}
squigg@amazon.com
Acct ID: 123456789012
Authenticate with
squigg access keys
Optionally also with MFA
Get temporary
security credentials
for ec2-role
Call AWS APIs
using temporary
security credentials
of ec2-role
{
"Statement":
[
{
"Effect":
"Allow",
"Action":
"sts:AssumeRole",
"Resource":
"arn:aws:iam::111122223333:role/ec2-­‐role"
}
]
}
{
"Statement":
[
{
"Effect":"Allow",
"Principal":{"AWS":"123456789012"},
"Action":"sts:AssumeRole"
}
]
}
Cross-account API access
ec2-role trusts IAM users from the AWS account
squigg@amazon.com (123456789012)
Permissions assigned to squigg granting him permission
to assume ec2-role in dsamuel@amazon.com account
IAM user: squigg
Permissions assigned
to ec2-role
STS
Amazon EC2

23. How to Keep Up to Date
• AWS Podcast
– http://aws.amazon.com/podcasts/aws-podcast/
• Amazon Web Services Blog
– http://aws.amazon.com/blogs/aws
• What’s New from AWS
– http://aws.amazon.com/new
• Social Media
– @awscloud, /amazonwebservices, /amazonwebservices
• Your Friendly Solution Architect Team
– Speak to the team today at the SA booth

24. Expand your skills with AWS
Certification
aws.amazon.com/certification
Exams
Validate your proven
technical expertise with
the AWS platform
On-Demand
Resources
aws.amazon.com/training/
self-paced-labs
Videos & Labs
Get hands-on practice
working with AWS
technologies in a live
environment
aws.amazon.com/training
Instructor-Led
Courses
Training Classes
Expand your technical
expertise to design, deploy,
and operate scalable,
efficient applications on AWS