Amazon EC2 forms the backbone compute platform for hundreds of thousands of AWS customers, but how do you go beyond starting an instance and manually configuring it?
The topics covered in this presentation include:
• Understanding how to use Amazon EC2 beyond a simple single instance use case
• Learn about instance bootstrapping, AMIs and Elastic IPs
• Learn how to optimise your costs by combining On Demand, Reserved and Spot Instances
• Discover how to create an Elastic Load Balancer and integrate it with Auto Scaling
• Learn how to create Auto Scaling configurations and the tools you need to drive Auto Scaling policies
• Learn how to use CloudWatch to alarms to trigger actions with Auto Scaling
• Discover additional resources that you can access to learn more
2. A technical deep dive beyond the basics
Help educate you on how to get the best from AWS technologies
Show you how things work and how to get things done
Broaden your knowledge in ~45 mins
Masterclass
3. On-demand compute to run application workloads
Easy come easy go – disposable resource
We provide the infrastructure, you decide what you run
Amazon EC2
4. What is EC2?
Elastic capacity Flexible
Complete control
Reliable
Inexpensive
Secure
28. AMIs
Your machine images
AMIs you have created from
EC2 instances
Can be kept private or shared
with other accounts
Amazon maintained
Set of Linux and Windows
images
Kept up to date by Amazon in
each region
Community
maintained
Images published by other AWS
users
Managed and maintained by
Marketplace partners
29. WindowsLinux Enterprise Linux
Small instance from
$0.060 per hour
Small instance from
$0.091 per hour
AMIs
Small instance from
$0.120 per hour
Small instance from
$0.090 per hour
Find out more at : aws.amazon.com/ec2/pricing
30. Unix/Linux instances start at
$0.02/hour
Pay as you go for compute power
Low cost and flexibility
Pay only for what you use, no up-front
commitments or long-term contracts
Use Cases:
Applications with short term, spiky, or
unpredictable workloads;
Application development or testing
On-demand instances
Instance types
31. Unix/Linux instances start at
$0.02/hour
Pay as you go for compute power
Low cost and flexibility
Pay only for what you use, no up-front
commitments or long-term contracts
Use Cases:
Applications with short term, spiky, or
unpredictable workloads;
Application development or testing
On-demand instances
1- or 3-year terms
Pay low up-front fee, receive significant hourly
discount
Low Cost / Predictability
Helps ensure compute capacity is available
when needed
Use Cases:
Applications with steady state or predictable
usage
Applications that require reserved
capacity, including disaster recovery
Reserved instances
Instance types
32. Unix/Linux instances start at
$0.02/hour
Pay as you go for compute power
Low cost and flexibility
Pay only for what you use, no up-front
commitments or long-term contracts
Use Cases:
Applications with short term, spiky, or
unpredictable workloads;
Application development or testing
On-demand instances
1- or 3-year terms
Pay low up-front fee, receive significant hourly
discount
Low Cost / Predictability
Helps ensure compute capacity is available
when needed
Use Cases:
Applications with steady state or predictable
usage
Applications that require reserved
capacity, including disaster recovery
Reserved instances
> 80% utilization
Lower costs up to 58%
Use Cases: Databases, Large Scale
HPC, Always-on infrastructure, Baseline
Heavy utilization RIInstance types
33. Unix/Linux instances start at
$0.02/hour
Pay as you go for compute power
Low cost and flexibility
Pay only for what you use, no up-front
commitments or long-term contracts
Use Cases:
Applications with short term, spiky, or
unpredictable workloads;
Application development or testing
On-demand instances
1- or 3-year terms
Pay low up-front fee, receive significant hourly
discount
Low Cost / Predictability
Helps ensure compute capacity is available
when needed
Use Cases:
Applications with steady state or predictable
usage
Applications that require reserved
capacity, including disaster recovery
Reserved instances
> 80% utilization
Lower costs up to 58%
Use Cases: Databases, Large Scale
HPC, Always-on infrastructure, Baseline
Heavy utilization RI
41-79% utilization
Lower costs up to 49%
Use Cases: Web applications, many heavy
processing tasks, running much of the time
Medium utilization RI
Instance types
34. Unix/Linux instances start at
$0.02/hour
Pay as you go for compute power
Low cost and flexibility
Pay only for what you use, no up-front
commitments or long-term contracts
Use Cases:
Applications with short term, spiky, or
unpredictable workloads;
Application development or testing
On-demand instances
1- or 3-year terms
Pay low up-front fee, receive significant hourly
discount
Low Cost / Predictability
Helps ensure compute capacity is available
when needed
Use Cases:
Applications with steady state or predictable
usage
Applications that require reserved
capacity, including disaster recovery
Reserved instances
> 80% utilization
Lower costs up to 58%
Use Cases: Databases, Large Scale
HPC, Always-on infrastructure, Baseline
Heavy utilization RI
41-79% utilization
Lower costs up to 49%
Use Cases: Web applications, many heavy
processing tasks, running much of the time
Medium utilization RI
15-40% utilization
Lower costs up to 34%
Use Cases: Disaster Recovery, Weekly /
Monthly reporting, Elastic Map Reduce
Light utilization RI
Instance types
35. Unix/Linux instances start at
$0.02/hour
Pay as you go for compute power
Low cost and flexibility
Pay only for what you use, no up-front
commitments or long-term contracts
Use Cases:
Applications with short term, spiky, or
unpredictable workloads;
Application development or testing
On-demand instances
1- or 3-year terms
Pay low up-front fee, receive significant hourly
discount
Low Cost / Predictability
Helps ensure compute capacity is available
when needed
Use Cases:
Applications with steady state or predictable
usage
Applications that require reserved
capacity, including disaster recovery
Reserved instances
Bid on unused EC2 capacity
Spot Price based on
supply/demand, determined automatically
Cost / Large Scale, dynamic workload handling
Use Cases:
Applications with flexible start and end times
Applications only feasible at very low compute
prices
Spot instances
Instance types
40. Public Key
Inserted by Amazon into
each EC2 instance that you
launch
Private Key
Downloaded and stored by
you
EC2
Instance
Comms secured
with private key
41. x.509Keypairs Credentials
Used to authenticate
when accessing and
instance
Used to authenticate
against some APIs
Keypairs & Secrets
Access key and secret
key used to authenticate
against APIs
48. IAM Roles and EC2 tools
1. Start an EC2 Linux instance
2. Assign an IAM role at launch time:
3. Sets up all the tools you need & manages
API access credentials
1. Up and running with CLI tools in a couple
of minutes – just SSH on and use
2. Terminate/stop instance when you are
done
{
"Statement": [
{
"Effect": "Allow",
"NotAction": "iam:*",
"Resource": "*"
}
]
}
55. Bake an AMI
Start an instance
Configure the instance
Create an AMI from your
instance
Start new ones from the
AMI
Bootstrapping
56. Bake an AMI Configure dynamically
Start an instance
Configure the instance
Create an AMI from your
instance
Start new ones from the
AMI
Bootstrapping
Launch an instance
Use metadata service
and cloud-init to perform
actions on instance
when it launches
vs
57. Bake an AMI Configure dynamically
Build your base images
and setup custom
initialisation scripts
Maintain your ‘golden’
base
Bootstrapping
Use bootstrapping to
pass custom information
in and perform post
launch tasks like pulling
code from SVN
+
58. Bake an AMI Configure dynamically
Bootstrapping
Time consuming
configuration (startup time)
Static configurations (less
change management)
59. Bake an AMI Configure dynamically
Bootstrapping
Continuous deployment
(latest code)
Environment specific (dev-
test-prod)
60. Goal is bring an instance up in a
useful state
The balance will vary depending upon your
application
64. #!/bin/sh
yum -y install httpd php mysql php-mysql
chkconfig httpd on
/etc/init.d/httpd start
Shell script in user-data will be executed on launch:
65. 65
Amazon Windows EC2Config Service executes user-data on
launch:
<script>dir > c:test.log</script>
<powershell>any command that you can run</powershell>
<powershell>
Read-S3Object -BucketName myS3Bucket
-Key myFolder/myFile.zip
-File c:destinationFile.zip
</powershell>
AWS Powershell Tools (use IAM roles as before…)
66. Why do this?
Automation
Less fingers, less mistakes
Availability
Drive higher
availability with self-
healing
Security
Instances locked
down by default
Flexible
Shell, Powershell,
CloudFormation,C
hef, Puppet, Ops
Works Scale
Manage large scale
deployments and drive
autoscaling
Efficiency
Audit and manage
your estate with less
time & effort
67. Do
Some does and don’ts
Use IAM roles
Go keyless if you can
Strike a balance between
AMI and dynamic
bootstrapping
68. Do Don’t
Some does and don’ts
Use IAM roles
Go keyless if you can
Strike a balance between
AMI and dynamic
bootstrapping
Put your API access keys
into code (and then publish
to GIT) or bake into AMIs
(and share)
71. Elastic Block StorageInstance Storage VS
Local ‘on host’ disk
volumes
Data dependent upon
instance lifecycle
Network attached optimised
block storage
Data independent of instance
lifecycle
72. Instance Storage
Local ‘on host’ disk
volumes
Data dependent upon
instance lifecycle
Host 1
eph0 eph1 eph2 eph3
Instance Store
Instance A
Instance B
Instance C
Host 2
eph0 eph1 eph2 eph3
Instance Store
Instance D
Instance E
Instance F
73. Instance Storage
Local ‘on host’ disk
volumes
Data dependent upon
instance lifecycle
If an instance reboots (intentionally or
unintentionally), data in the instance store
persists
Data on instance store volumes is lost under the
following circumstances:
• Failure of an underlying drive
• Stopping an Amazon EBS-backed instance
• Terminating an instance
74. Options
Differing types of
instance storage
General purpose m3.medium 1 x 4 SSD*6
Compute optimized c1.xlarge 4 x 420
General purpose m3.large 1 x 32 SSD*6
Compute optimized cc2.8xlarge 4 x 840
General purpose m3.xlarge 2 x 40 SSD*6
GPU instances g2.2xlarge 1 x 60 SSD
General purpose m3.2xlarge 2 x 80 SSD*6
GPU instances cg1.4xlarge 2 x 840
General purpose m1.small 1 x 160 Memory optimized m2.xlarge 1 x 420
General purpose m1.medium 1 x 410 Memory optimized m2.2xlarge 1 x 850
General purpose m1.large 2 x 420 Memory optimized m2.4xlarge 2 x 840
General purpose m1.xlarge 4 x 420 Memory optimized cr1.8xlarge 2 x 120 SSD
Compute optimized c3.large 2 x 16 SSD Storage optimized i2.xlarge 1 x 800 SSD
Compute optimized c3.xlarge 2 x 40 SSD Storage optimized i2.2xlarge 2 x 800 SSD
Compute optimized c3.2xlarge 2 x 80 SSD Storage optimized i2.4xlarge 4 x 800 SSD
Compute optimized c3.4xlarge 2 x 160 SSD Storage optimized i2.8xlarge 8 x 800 SSD
Compute optimized c3.8xlarge 2 x 320 SSD Storage optimized hs1.8xlarge 24 x 2,048*3
Compute optimized c1.medium 1 x 350 Storage optimized hi1.4xlarge 2 x 1,024 SSD
Instance Storage
75. Elastic Block Storage
Network attached optimised
block storage
Data independent of instance
lifecycle
EBSEC2
Workspace
Hypervisor
S3
EBS
snapshot
Network
One or more ephemeral
(temporary) drives
(instance storage)
One or more EBS
(persistent) drives
EBS snapshots
(backup images)
80. EBS Persistence
EBS volume is off-instance storage
You pay for the volume usage as long as the data
persists
1. By default, EBS volumes that are attached to a running instance
automatically detach from the instance with their data intact when that
instance is terminated
2. By default, EBS volumes that are created and attached to an instance
at launch are deleted when that instance is terminated. You can modify
this behavior by changing the value of the flag DeleteOnTermination to
false when you launch the instance.
83. Availability Zone Availability Zone
Region
Availability Zone
Instance Instance Instance Instance Instance Instance
Elastic Load Balancer
84. Offload
SSL processing on ELB
Remove load from EC2
instances
Spread
Go small and wide
Balance resources across
AZs
Health check
Choose the right healthcheck
point
Check whole layers
Elastic Load Balancing
85. 1. Persistent HTTP connections – enable them and ELB to
Server will be optimized
2. Never address underlying IP – always DNS name
• There’s a set behind an ELB and real clients spread
across them
• They will change as the ELB scales to keep ahead of
demand
3. If you span ELB across AZs have an instance in all Azs
4. De-register instances from an ELB before terminating
88. Describes what Auto Scaling
will create when adding
Instances
AMI
Instance Type
Security Group
Instance Key Pair
Only one active launch configuration
at a time
Auto Scaling will terminate instances
with old launch configuration first
rolling update
Auto Scaling managed grouping
of EC2 instances
Automatic health check to maintain
pool size
Automatically scale the number of
instances by policy –
Min, Max, Desired
Automatic Integration with ELB
Automatic distribution & balancing
across AZs
Parameters for performing an
Auto Scaling action
Scale Up/Down and by how much
ChangeInCapacity (+/- #)
ExactCapacity (#)
ChangeInPercent (+/- %)
Cool Down (seconds)
Policy can be triggered by
CloudWatch events
Launch Configuration Auto-Scaling Group Auto-Scaling Policy
95. Create an auto-scaling policy (scale up):
Period before another action will take place
(Damper)
aws autoscaling put-scaling-policy
--policy-name 101ScaleUpPolicy
--auto-scaling-group-name 101-as-group
--scaling-adjustment 1
--adjustment-type ChangeInCapacity
--cooldown 300
107. Route 53
Front EC2 and ELBs with
Route 53 for control over
DNS
Resource tagging
Tag resources like EC2
and have it appear on
billing reports
Rolling deployments
Use Route 53 and ELBs to do
rolling deployments, A/B
testing
Other topics…
108. OpsWorks
Manage stacks as layers
and implement Chef recipes
to automate EC2
configuration
Beanstalk
Manage an entire
autoscaling stack for
popular containers such
as ruby, python etc
CloudFormation
Template everything from
configuration of CloudWatch
alarms, SNS topics, EC2
instances
Other topics…
110. Stop doing these:
Provisioning and fixing servers
Treating compute as physical things
Thinking of compute as a finite commitment
111. and start doing these
Security
Build systems secure by
default
Elasticity
Stateless autoscaling
applications
Replace not fix
Build from scratch, don’t fix
something Unconstrained
Say goodbye to traditional
capacity planning
Be cost aware
Tag resources, play with
instance types
Automation
Create instances when
you need them, drop
them when not
113. Join us and learn about the AWS
platform, new services and how to get
started.
Register for a Summit near you.
aws.amazon.com/aws-summit-2014/
San Francisco | March 26
Register now
Sydney | April 9
Register now
London | April 30
Register now
Auckland | May 7
Paris | May 13
Berlin | May 15
Register now
Stockholm | May 22
Sao Paulo | May 27
Amsterdam | June 10
New York | July 10
Tokyo | July 17-18
Beijing | TBD
Seoul | September 3
Tel Aviv | September 17
115. AWS Training & Certification
Certification
aws.amazon.com/certification
Demonstrate your skills,
knowledge, and expertise
with the AWS platform
Self-Paced Labs
aws.amazon.com/training/
self-paced-labs
Try products, gain new
skills, and get hands-on
practice working with
AWS technologies
aws.amazon.com/training
Training
Skill up and gain
confidence to
design, develop, deploy
and manage your
applications on AWS
116. We typically see customers start by trying our
services
Get started now at : aws.amazon.com/getting-started
117. Design your application for the AWS Cloud
More details on the AWS Architecture Center at : aws.amazon.com/architecture
Notas del editor
45 minutes is not a lot of time to cover EC2, so if there’s anything we don’t cover that you would like to know more about, then ask a question and we will get back to you with the answer, or point you to some resources that can help you learn more.
You have complete root or administrator control over your stackYou can grow from 1 to many thousands of servers on demandSecurity is a priority for AWS and is built into the stackIt’s very inexpensive to start with small instances and you can scale up very easily if you need to
Every customer is securely segregated and we go to great lengths to ensure that customers are isolated from one another – including external audit
An AMI is a golden master of a virtual machine.
An instance is a running or stopped virtual machine that has been created from an AMI
The VPC gives you control over networking constructs such as addressing, but we are not going to cover this in detail today.
Each availability zone within a region is an independent failure domain with a region
You can launch your instances in multiple availability zones and distribute your applications across these multiple failure domains to achieve high levels of availability for your applications.
Within an availability zone we have something called Elastic Block Store. This is a disk drive that underpins your EC2 instances and we will drill down into this later in this webinar. These EBS volumes are redundant within an availability zone.
S3, our simple storage service, provides a capability to reliably store snapshots of your EBS disk drives within your region and recall these should you need them.
The instance should be considered your unit of control, your unit of scale and your unit of resilience.
You can bundle your stack into an instance and make this your unit of control. Determining later the region that you will run this in, the size and type of instance that you will run it on and when you will chose to destroy your instance. §
You can scale out very easily in EC2. You can also play with cost dynamics, using a large fleet of smaller instances, which can both reduce your costs and improve your resilience
As you spread your workload out
The core things that you need to know when launching an instance are here:Where you want to launch it, such as EU WestThe instance size that you wish to launch, such as the m1.smallThe Amazon Machine Image that you are going to launch from, one of ours or one of yoursKey pair – authentication for a human to log onSecurity group - Firewalling around an instance to control trafficThe last two are important for securing your instance, so let’s look at these in some more detail.
Here’s how the boot cycle works for your EC2 instanceFirst we will identify or start the hypervisor workspace for the instance that we are going to create
Next we attach the instance storage to that workspace, which provides the ephemeral storage that we talked about earlier
Then we create a device for the boot volume from an AMI that is stored in S3
Lastly we connect this new volume to your instance and start the instance, booting it from this newly created volume
Thinking about the EBS persistence cycle, you only pay for a volume for the length of time that it persists in your account.You need to be aware of these two default lifecycles, and the fact that you can change the default for volumes that are created at instance launch in order to make them persist.
You have the optionsubsystento define a custom health check.
Speaker Notes:We have several programs available to help you deepen your knowledge and proficiency with AWS. We encourage you to check out the following resources:Get hands-on experience testing products and gaining practical experience working with AWS technologies by taking an AWS Self-Paced Lab at run.qwiklab.com. Available anywhere, anytime, you have freedom to take self-paced labs on-demand and learn at your own pace. AWS self-paced labs were designed by AWS subject matter experts and provide an opportunity to use the AWS console in a variety of pre-designed scenarios and common use cases, giving you hands-on practice in a live AWS environment to help you gain confidence working with AWS. You have flexibility to choose from topics and products about which you want to learn more.Take an instructor-led AWS Training course. We have a variety of role-based courses to meet the requirements of your job role and business need, whether you’re a Solutions Architect, Developer, SysOps Administrator, or just interested in learning AWS fundamentals.AWS Certification validates your skills, knowledge an expertise in working with AWS services. Earning certification enables you to gain visibility and credibility for your skills.
750 hours for EC2 per month, for Linux and Windows, 5Gb for S3, and 30Gb for EBS
If you already have a project in mind then you may just want to start by designing the application.