Learning Objectives: - Learn how Amazon EC2 Spot Instances can help run and scale High Performance Computing (HPC) workload for a fraction of the cost - Learn how to deploy an HPC cluster on EC2 Spot Instances using standard HPC tools such as schedulers, shared storage, and an MPI environment - Leverage the full scale of the AWS cloud for faster results

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Chad Schmutzer, Solutions Architect – EC2 Spot
Instances
December 5th, 2017
Run Your HPC Workload at
Scale for a Fraction of the Cost

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What Are We Going to Do Today?
• Learn about Amazon EC2 Spot Instances
• Learn about EC2 Spot Instances best practices
• Understand tools for powering your HPC workloads with
EC2 Spot Instances

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
On-Demand
Pay for compute
capacity by the hour
with no long-term
commitments
For spiky workloads,
or to define needs
AWS EC2 Consumption Models
Reserved
Make a low, one-time
payment and receive
a significant discount
on the hourly charge
For committed
utilization
Spot Market
Access unused
capacity, charged at a
Spot Price which slowly
fluctuates based on
supply and demand
For time-insensitive,
transient, or stateless
workloads

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Spare Capacity at Scale
AWS has over a million active
customers in 190 countries,
supporting virtually any
workload.

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Spare Capacity at Scale
AWS has over a million active
customers in 190 countries,
supporting virtually any
workload.
In other words, there is more
than enough for everyone.

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What Are EC2 Spot Instances?
EC2 Spot Instances are
spare EC2 On-Demand capacity
with very simple rules…

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What Are EC2 Spot Instances?
EC2 Spot Instances are
spare EC2 On-Demand capacity
with very simple rules…

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The Very Simple Rules of Spot
Instances

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The Very Simple Rules of Spot Instances
Run in markets where the
price of compute slowly
changes based on supply
and demand.

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The Very Simple Rules of Spot Instances
Run in markets where the price of
compute changes based on supply
and demand.
You control your spend. When the
market exceeds what you are
willing to pay, you get 2 minutes to
wrap up your work.

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Get the Best Value for EC2 Capacity
Since Spot Instances typically cost 50-90% less than On-
Demand, you can:

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Get the Best Value for EC2 Capacity
Since Spot Instances typically cost 50-90% less than On-
Demand, you can:
• Increase your compute capacity by 2-10x within the same
budget

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Get the Best Value for EC2 Capacity
Since Spot Instances typically cost 50-90% less than On-
Demand, you can:
• Increase your compute capacity by 2-10x within the same
budget
• Save 50-90% on your existing workload

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Get the Best Value for EC2 Capacity
Since Spot Instances typically cost 50-90% less than On-
Demand, you can:
• Increase your compute capacity by 2-10x within the same
budget
• Save 50-90% on your existing workload
• Or both!

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Get the Best Value for EC2 Capacity
Since Spot Instances typically cost 50-90% less than On-
Demand, you can:
• Increase your compute capacity by 2-10x within the same
budget
• Save 50-90% on your existing workload
• Or both!
Either way, you should try it!

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Understanding EC2 Capacity
AZ1
AZ2
(N. California) Total Capacity
P3 C4 M4 I3 R4 D2
Shared
Dedicated
Shared
Dedicated
x 2x 4x x 2x 4x x 2x 4x x 2x 4x x 2x 4x x 2x 4x

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
So just how much spare
capacity is there?

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Understanding EC2 Capacity
• On average, every week, AWS customers are using more compute
capacity on Amazon EC2 Spot Instances than customers in 2012
were running across all of Amazon EC2.

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Understanding EC2 Capacity
• On average, every week, AWS customers are using more compute
capacity on Amazon EC2 Spot Instances than customers in 2012
were running across all of Amazon EC2.
• A group of researchers from Clemson University achieved a
remarkable milestone while studying topic modeling, breaking the
record for creating the largest high-performance cluster in the cloud
by using more than 1,100,000 vCPUs on Amazon EC2 Spot
Instances running in a single AWS region.

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
1,119,196 vCPUs
http://amzn.to/2wmYkWm

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
1,119,196 vCPUs
http://amzn.to/2wmYkWm

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
$0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
Capacity and Spot Markets Recap
• Each instance family
us-east-2

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
$0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
Capacity and Spot Markets Recap
• Each instance family
• Each instance size
us-east-2

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
$0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
Capacity and Spot Markets Recap
• Each instance family
• Each instance size
• In each Availability Zone
us-east-2

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
$0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
Capacity and Spot Markets Recap
• Each instance family
• Each instance size
• In each Availability Zone
• In every Region
us-east-2

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
$0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
Capacity and Spot Markets Recap
• Each instance family
• Each instance size
• In each Availability Zone
• In every Region
• Is a separate pool of
capacity and therefore a
separate Spot Market
us-east-2

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2 Spot Instance Pricing
• With Spot instances,
you will never be
charged more than
the maximum price
you specified.

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2 Spot Instance Pricing
• With Spot instances,
you will never be
charged more than
the maximum price
you specified.
• While your instance
runs, you are
charged the Spot
price that is in effect
for that period.
You pay the
market
price

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2 Spot Instance Best Practices - Flexibility
Fault toleranceStateless Multi-AZ
Loosely
coupled

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2 Spot Instance Best Practices - Flexibility
Fault toleranceStateless Multi-AZ
Loosely
coupled
Instance
Flexibility

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2 Spot Instance Best Practices - Diversification
• Multiple instance types selected
• Multiple Availability Zones selected
• Pick instance types with similar performance
characteristics. For example: c4.large, r4.large, m4.large

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Using a single
additional parameter
Run continuously
for up to 6 hours
Save up to 50% off
On-Demand pricing
EC2 Spot Blocks - Defined Duration
$1

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2 Spot Fleet
A single API call to help you…

34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Spot Fleet Helps You…
Launch Thousands of Spot Instances
with one RequestSpotFleet API call
Get Best Price
Find the lowest priced horsepower that works for you
or
Get Diversified Resources
Diversify your fleet – increase your availability
Apply Custom Weighting
Create your own capacity unit based on your application
needs

35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Spot Fleet Helps You…
Maintain Target Capacity
Take Advantage of Automatic Scaling
Tagging, Health Checks, CloudWatch
Metrics

36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What Types of Workloads Are Customers Running
on Spot Instances?
• Queue and Batch based processing
• High Performance Computing
• Hadoop / Spark / Big Data workloads (EMR)
• Stateless Applications (e.g. web tiers)
• CI/CD Pipelines
• Containerized workloads powered by Spot Fleet
• Live / Video On-Demand Streaming Content

37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What Types of Workloads Are Customers Running
on Spot Instances?
• Queue and Batch based processing
• High Performance Computing
• Hadoop / Spark / Big Data workloads (EMR)
• Stateless Applications (e.g. web tiers)
• CI/CD Pipelines
• Containerized workloads powered by Spot Fleet
• Live / Video On-Demand Streaming Content

38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Demo
• CfnCluster
• AWS Batch

39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
https://aws.amazon.com/ec2/spot/
https://github.com/awslabs/ec2-spot-labs
https://wiki.jenkins.io/display/JENKINS/Amazon+EC2+Fleet+Plugin
Thank You!