Developing applications on Amazon Web Services (AWS) or moving your business into the cloud is more straightforward than you think. Whether you are a developer eager to learn new skills, a solutions architect who wants to solve existing technology problems, the IT professional who wants access to cost-effective, on-demand computing resources, this workshop is for you.
These slides feature some of the most popular Amazon Web Services: Amazon Elastic Compute Service (EC2), Amazon Simple Storage Service (S3), Amazon CloudFront, Amazon Elastic Block Storage (EBS) and Amazon Relational Database Service (RDS).
Amazon EC2 YouTube Recording: http://youtu.be/TORzO9Oc9oU
Amazon EC2 Demo: http://youtu.be/kMExnVKhmYc
2. Masterclass
A technical deep dive beyond the basics
Help educate you on how to get the best from AWS technologies
Show you how things work and how to get things done
Broaden your knowledge in ~45 mins
3. Amazon EC2
On-demand compute to run application workloads
Easy come easy go – disposable resource
We provide the infrastructure, you decide what you run
25. AMIs
Amazon Community Your machine
maintained maintained images
Set of Linux and Windows Images published by other AMIs you have created from
images AWS users EC2 instances
Kept up to date by Amazon Managed and maintained by Can be kept private or shared
in each region Marketplace partners with other accounts
27. AMIs
Linux Enterprise Linux Windows
Small instance from Small instance from Small instance from
$0.060 per hour $0.120 per hour $0.115 per hour
Small instance from
$0.090 per hour
28. Instance types
On-demand instances
Unix/Linux instances start at
$0.02/hour
Pay as you go for compute power
Low cost and flexibility
Pay only for what you use, no up-front
commitments or long-term contracts
Use Cases:
Applications with short term, spiky, or
unpredictable workloads;
Application development or testing
29. Instance types
On-demand instances Reserved instances
Unix/Linux instances start at 1- or 3-year terms
$0.02/hour
Pay low up-front fee, receive significant hourly
Pay as you go for compute power discount
Low cost and flexibility Low Cost / Predictability
Pay only for what you use, no up-front Helps ensure compute capacity is available
commitments or long-term contracts when needed
Use Cases:
Use Cases:
Applications with short term, spiky, or
unpredictable workloads; Applications with steady state or predictable
usage
Application development or testing
Applications that require reserved capacity,
including disaster recovery
30. Instance types Heavy utilization RI
> 80% utilization
Lower costs up to 58%
On-demand instances Reserved instances Use Cases: Databases, Large Scale HPC,
Always-on infrastructure, Baseline
Unix/Linux instances start at 1- or 3-year terms
$0.02/hour
Pay low up-front fee, receive significant hourly
Pay as you go for compute power discount
Low cost and flexibility Low Cost / Predictability
Pay only for what you use, no up-front Helps ensure compute capacity is available
commitments or long-term contracts when needed
Use Cases:
Use Cases:
Applications with short term, spiky, or
unpredictable workloads; Applications with steady state or predictable
usage
Application development or testing
Applications that require reserved capacity,
including disaster recovery
31. Instance types Heavy utilization RI
> 80% utilization
Lower costs up to 58%
On-demand instances Reserved instances Use Cases: Databases, Large Scale HPC,
Always-on infrastructure, Baseline
Unix/Linux instances start at 1- or 3-year terms
$0.02/hour
Pay low up-front fee, receive significant hourly Medium utilization RI
Pay as you go for compute power discount
Low cost and flexibility Low Cost / Predictability 41-79% utilization
Lower costs up to 49%
Pay only for what you use, no up-front Helps ensure compute capacity is available Use Cases: Web applications, many heavy
commitments or long-term contracts when needed processing tasks, running much of the time
Use Cases:
Use Cases:
Applications with short term, spiky, or
unpredictable workloads; Applications with steady state or predictable
usage
Application development or testing
Applications that require reserved capacity,
including disaster recovery
32. Instance types Heavy utilization RI
> 80% utilization
Lower costs up to 58%
On-demand instances Reserved instances Use Cases: Databases, Large Scale HPC,
Always-on infrastructure, Baseline
Unix/Linux instances start at 1- or 3-year terms
$0.02/hour
Pay low up-front fee, receive significant hourly Medium utilization RI
Pay as you go for compute power discount
Low cost and flexibility Low Cost / Predictability 41-79% utilization
Lower costs up to 49%
Pay only for what you use, no up-front Helps ensure compute capacity is available Use Cases: Web applications, many heavy
commitments or long-term contracts when needed processing tasks, running much of the time
Use Cases:
Use Cases:
Applications with short term, spiky, or Light utilization RI
unpredictable workloads; Applications with steady state or predictable
usage
Application development or testing 15-40% utilization
Applications that require reserved capacity, Lower costs up to 34%
including disaster recovery
Use Cases: Disaster Recovery, Weekly /
Monthly reporting, Elastic Map Reduce
33. Instance types
On-demand instances Reserved instances Spot instances
Unix/Linux instances start at 1- or 3-year terms Bid on unused EC2 capacity
$0.02/hour
Pay low up-front fee, receive significant hourly Spot Price based on supply/demand,
Pay as you go for compute power discount determined automatically
Low cost and flexibility Low Cost / Predictability Cost / Large Scale, dynamic workload handling
Pay only for what you use, no up-front Helps ensure compute capacity is available
commitments or long-term contracts when needed
Use Cases:
Use Cases:
Use Cases: Applications with flexible start and end times
Applications with short term, spiky, or
unpredictable workloads; Applications with steady state or predictable Applications only feasible at very low compute
usage prices
Application development or testing
Applications that require reserved capacity,
including disaster recovery
38. Public Key
Inserted by Amazon into
each EC2 instance that
you launch
EC2
Instance
Comms secured
with private key
Private Key
Downloaded and stored
by you
39. Keypairs & Secrets
Keypairs Credentials x.509
Used to authenticate Access key and secret key Used to authenticate
when accessing and used to authenticate against some APIs
instance against APIs
46. IAM Roles and EC2 tools
1. Start an EC2 Linux instance
2. Assign an IAM role at launch time:
3. Sets up all the tools you need & manages
{
API access credentials "Statement": [
{
1. Up and running with CLI tools in a couple "Effect": "Allow",
"NotAction": "iam:*",
of minutes – just SSH on and use
"Resource": "*"
}
2. Terminate/stop instance when you are ]
done }
52. Instances don’t need keypairs
But how do you configure it if you can’t log
onto it?
53. Bootstrapping
Bake an AMI
Start an instance
Configure the instance
Create an AMI from
your instance
Start new ones from
the AMI
54. Bootstrapping
Bake an AMI vs Configure dynamically
Start an instance Launch an instance
Configure the instance Use metadata service
and cloud-init to
Create an AMI from
perform actions on
your instance
instance when it
Start new ones from launches
the AMI
55. Bootstrapping
Bake an AMI + Configure dynamically
Build your base images Use bootstrapping to
and setup custom pass custom
initialisation scripts information in and
perform post launch
Maintain your ‘golden’
tasks like pulling code
base
from SVN
56. Bootstrapping
Bake an AMI Configure dynamically
Time consuming
configuration (startup time)
Static configurations (less
change management)
57. Bootstrapping
Bake an AMI Configure dynamically
Continuous deployment
(latest code)
Environment specific (dev-
test-prod)
58. Goal is bring an instance up in a
useful state
The balance will vary depending upon your application
61. Instance User Meta-data
request data service
Instance
62. Shell script in user-data will be executed on launch:
#!/bin/sh
yum -y install httpd php mysql php-mysql
chkconfig httpd on
/etc/init.d/httpd start
63. Amazon Windows EC2Config Service executes user-
data on launch:
<script>dir > c:test.log</script>
<powershell>any command that you can run</powershell>
AWS Powershell Tools (use IAM roles as before…)
<powershell>
Read-S3Object -BucketName myS3Bucket
-Key myFolder/myFile.zip
-File c:destinationFile.zip
</powershell>
63
64. Automation
Less fingers, less mistakes
Security Availability
Instances locked
Drive higher
down by default
availability with self-
healing
Why do this?
Flexible Efficiency
Shell, Powershell, Audit and manage
CloudFormation, your estate with less
Chef, Puppet, Scale time & effort
OpsWorks Manage large scale
deployments and drive
autoscaling
65. Some does and don’ts
Do
Use IAM roles
Go keyless if you can
Strike a balance between
AMI and dynamic
bootstrapping
66. Some does and don’ts
Do Don’t
Use IAM roles Put your API access keys
into code (and then publish
Go keyless if you can
to GIT) or bake into AMIs
Strike a balance between (and share)
AMI and dynamic
bootstrapping
69. Instance Storage VS Elastic Block Storage
Local ‘on host’ disk Network attached optimised
volumes block storage
Data dependent upon Data independent of
instance lifecycle instance lifecycle
70. Instance A
Instance Storage Instance D
Instance B
Local ‘on host’ disk
volumes Instance E
Instance C
Data dependent upon Instance F
instance lifecycle Instance Store Instance Store
eph0 eph1 eph2 eph3 eph0 eph1 eph2 eph3
Host 1 Host 2
71. If an instance reboots (intentionally or
Instance Storage unintentionally), data in the instance store
persists
Local ‘on host’ disk
volumes
Data on instance store volumes is lost under
Data dependent upon the following circumstances:
instance lifecycle
• Failure of an underlying drive
• Stopping an Amazon EBS-backed instance
• Terminating an instance
74. One or more ephemeral
(temporary) drives
(instance storage)
One or more EBS
(persistent) drives
EBS snapshots
(backup images)
Elastic Block Storage
Network attached optimised
Workspace block storage
Network
Data independent of
EBS
snapshot
instance lifecycle
Hypervisor
EC2 EBS S3
79. EBS Persistence
EBS volume is off-instance storage
You pay for the volume usage as long as the data
persists
1. By default, EBS volumes that are attached to a running instance
automatically detach from the instance with their data intact when
that instance is terminated
2. By default, EBS volumes that are created and attached to an instance
at launch are deleted when that instance is terminated. You can
modify this behavior by changing the value of the flag
DeleteOnTermination to false when you launch the instance.
82. Elastic Load Balancer
Instance Instance Instance Instance Instance Instance
Availability Zone Availability Zone Availability Zone
Region
83. Elastic Load Balancing
Spread Offload Health check
Go small and wide SSL processing on ELB Choose the right healthcheck
Balance resources across Remove load from EC2 point
AZs instances Check whole layers
84. 1. Persistent HTTP connections – enable them and ELB
to Server will be optimized
2. Never address underlying IP – always DNS name
• There’s a set behind an ELB and real clients spread
across them
• They will change as the ELB scales to keep ahead
of demand
3. If you span ELB across AZs have an instance in all Azs
4. De-register instances from an ELB before terminating
87. Launch Configuration Auto-Scaling Group Auto-Scaling Policy
Describes what Auto Scaling Auto Scaling managed Parameters for performing an
will create when adding grouping of EC2 instances Auto Scaling action
Instances
Automatic health check to Scale Up/Down and by how much
AMI
maintain pool size
Instance Type ChangeInCapacity (+/- #)
Security Group ExactCapacity (#)
Automatically scale the number of
Instance Key Pair ChangeInPercent (+/- %)
instances by policy – Min, Max,
Desired
Only one active launch Cool Down (seconds)
configuration at a time
Automatic Integration with ELB
Policy can be triggered by
Auto Scaling will terminate CloudWatch events
Automatic distribution & balancing
instances with old launch
across AZs
configuration first
rolling update
94. Create an auto-scaling policy (scale up):
as-put-scaling-policy 101ScaleUpPolicy
--auto-scaling-group 101-as-group
--adjustment=1
--type ChangeInCapacity
--cooldown 300
Period before another action will take place
(Damper)
105. Other topics…
Resource tagging Route 53 Rolling deployments
Tag resources like EC2 Front EC2 and ELBs with Use Route 53 and ELBs to do
and have it appear on Route 53 for control over rolling deployments, A/B
billing reports DNS testing
106. Other topics…
Beanstalk OpsWorks CloudFormation
Manage an entire Manage stacks as layers Template everything from
autoscaling stack for and implement Chef configuration of CloudWatch
popular containers such recipes to automate EC2 alarms, SNS topics, EC2
as ruby, python etc configuration instances
108. Stop doing these:
Provisioning and fixing servers
Treating compute as physical things
Thinking of compute as a finite commitment
109. Elasticity
Stateless autoscaling Automation
Security applications Create instances when
Build systems secure by
you need them, drop
default
them when not
and start doing these
Replace not fix Be cost aware
Build from scratch, don’t Unconstrained Tag resources, play with
fix something instance types
Say goodbye to
traditional capacity
planning