Amazon Elastic Block Store (Amazon EBS) provides persistent block level storage for use with Amazon EC2 instances. In this technical session, we will present and demonstrate how you can increase capacity, tune performance, and modify volume types on the fly with the latest Amazon EBS innovation, Elastic Volumes. You will learn how Elastic Volumes can significantly reduce both operational complexity and downtime enabling you to right-size your deployment and dynamically adapt as your business needs change.
5. EBS volume types: I/O Provisioned
General Purpose
SSD
gp2
Throughput: 160 MiB/s
Latency: Single-digit ms
Capacity: 1 GiB to 16 TiB
Baseline: 3 IOPS per GiB up to 10,000
Burst: 3,000 IOPS (for volumes up to 1 TiB)
Great for boot volumes, low-latency applications, and bursty databases
6. EBS volume types: I/O Provisioned
Provisioned IOPS
SSD
io1
Baseline: 100 to 20,000 IOPS
Throughput: 320 MiB/s
Latency: Single-digit ms
Capacity: 4 GiB to 16 TiB
Max ratio: 50:1 IOPS to size
Ideal for critical applications and databases with sustained IOPS
7. EBS volume types: Throughput Provisioned
Throughput
Optimized HDD
st1
Ideal for large-block, high-throughput sequential workloads
Baseline: 40 MiB/s per TiB up to 500 MiB/s
Capacity: 500 GiB to 16 TiB
Burst: 250 MiB/s per TiB up to 500 MiB/s
8. Cold HDD
sc1
EBS volume types: Throughput Provisioned
Baseline: 12 MiB/s per TiB up to 192 MiB/s
Capacity: 500 GiB to 16 TiB
Burst: 80 MiB/s per TiB up to 250 MiB/s
Ideal for sequential throughput workloads, such as logging and backup
9. Throughput
is more important
Small, random I/O Large, sequential I/O
Latency?
i2
gp2 io1 st1sc1
d2
Choosing an EBS volume type
IOPS
≤ 65,000> 65,000
< 1 ms Single-digit ms ≤ 1,250 MiB/s
Aggregate throughput?
> 1,250 MiB/s
is more important
≤ 10k IOPS > 10k IOPS
Throughput per volume
250 MiB/s 500 MiB/s
IOPS per volume
27. EBS volume types: Throughput Provisioned
Throughput
Optimized HDD
st1
Baseline: 40 MiB/s per TiB up to 500 MiB/s
Capacity: 500 GiB to 16 TiB
Burst: 250 MiB/s per TiB up to 500 MiB/s
Ideal for large-block, high-throughput sequential workloads
32. Considerations
Volume modifications must be valid for the target
volume type
• Minimum volume sizes
for each volume type
• Maximum volume sizes
gp2
io1 st1
sc1
1 GiB
4 GiB
500 GiB
500 GiB
16
33. Considerations
Volume modifications must be valid for the target
volume type
• Perf. characteristics for
each volume type
• Maximum ratio of
PIOPS to size for io1
volumes
gp2
io1 st1
sc1
Min 100 iops
Max 3K iops
Min 100 iops
Max 20K iops
Min 12 MiB/s /TiB
Max 192 MiB/s /TiB
Min 40 MiB/s /TiB
Max 500 MiB/s /TiB
50:1
36. Where can I use it?
Available in all public AWS regions
Current generation volumes
Current generation instances*
* Live volume modifications
37. Available in all public AWS regions
N. Virginia
Ohio
N. California
Oregon
Canada
São Paulo
Ireland
London
Frankfurt
Sydney
Tokyo
Seoul
Mumbai
Singapore
Beijing
41. Caveats
M3 Instances other than Medium
M3
other than
Medium
= No Live Volume Modifications*
* After each volume modification you MUST stop/start the instance or
detach/attach the volume
42. Caveats
All other instances WITH
EBS volumes attached before 11/1/2016
All
EC2
= Live Volume Modifications*
* After a one-time instance stop/start or volume detach/attach
51. What can I expect?
No performance impact
No downtime
No over-provisioning
52. No performance impact
During the modification
there is no performance
impact as a result of the
process
Your volume performance will be between the original and
target volume performance characteristics
53. No performance impact
Your volume performance
will be between the
original and target volume
performance
characteristics during
modification.
Original
Volume
Target
Volume
Performance
54. No performance impact
Your volume performance
will be between the
original and target volume
performance
characteristics during
modification.
io1
gp2
1 TiB
1 TiB
example
55. No performance impact
Your volume performance
will be between the
original and target volume
performance
characteristics during
modification.
io1
gp2
3K iops
1 TiB
1 TiB
20K iops
example
56. No performance impact
Your volume performance
will be between the
original and target volume
performance
characteristics during
modification.
io1
gp2
3K iops
1 TiB
1 TiB
20K iops
example
Performance
57. No performance impact
Your volume performance
will be between the
original and target volume
performance
characteristics during
modification.
10 TiB
10 TiB
example
st1
io1
58. No performance impact
Your volume performance
will be between the
original and target volume
performance
characteristics during
modification.
20K iops
10 TiB
10 TiB
400 iops
example
st1
io1
59. No performance impact
Your volume performance
will be between the
original and target volume
performance
characteristics during
modification.
320 MiB/s
20K iops
10 TiB
10 TiB
400 iops
400 MiB/s
example
st1
io1
60. No performance impact
Your volume performance
will be between the
original and target volume
performance
characteristics during
modification.
320 MiB/s
20K iops
10 TiB
10 TiB
400 iops
400 MiB/s
example
Performance
st1
io1
84. State 2: Optimizing
Volume type modifications
Your volume performance will
be between the original and
target volume performance
characteristics during
modification.
85. State 2: Optimizing
Volume type modifications
Original
Volume
Target
Volume
Performance
Your volume performance will
be between the original and
target volume performance
characteristics during
modification.
86. State 2: Optimizing
Increase/decrease PIOPS
Original
Volume
Target
Volume
Performance
Your volume performance will
be between the original and
target volume performance
characteristics during
modification.
87. State 2: Optimizing
Increase volume size
As soon as you enter the
‘Optimizing’ state, your volume
is the target size
Go to ’Step 3’ & extend the file
system in the OS gp2
gp2
90. Step 3: Extend (only if increasing size)
Using EC2 instance OS
Linux:
Determine the file system (if unsure)
[ec2-user ~]$ sudo file –s /dev/xvd*
/dev/xvda1: Linux rev 1.0 ext4 filesystem data ...
/dev/xvdf: SGI XFS filesystem data ...
91. Step 3: Extend (only if increasing size)
Using EC2 instance OS
Linux:
Compare block device size to the file system disk usage
[ec2-user ~]$ lsblk
[ec2-user ~]$ df -h
92. Step 3: Extend (only if increasing size)
Using EC2 instance OS
Linux:
For ext2, ext3, ext4 file systems, use resize2fs
[ec2-user ~]$ sudo resize2fs device_name
For XFS file systems, use xfs_growfs
[ec2-user ~]$ sudo xfs_growfs –d mount_point
93. Step 3: Extend (only if increasing size)
Using EC2 instance OS
Windows:
Run diskmgmt.msc
94. Step 3: Extend (only if increasing size)
Using EC2 instance OS
Windows:
Right click the
expanded drive and
choose Extend
Volume.
95. Step 3: Extend (only if increasing size)
Using EC2 instance OS
Windows:
Enter the number of
MB to extend the
volume and
complete the wizard.
96. How often can I modify volumes?
Each volume can be
modified once every 6
hours
6hrs.
97. How often can I modify volumes?
Each volume can be
modified once every 6
hours
An error occurred (IncorrectState) when
calling the ModifyVolume operation:
You've reached the maximum modification
rate per volume limit. Wait at least 6
hours between modifications per EBS
volume.
98. Can I stop a volume modification once started?
A modify volume
command cannot be
stopped
99. Can I modify encrypted volumes?
Yes, modifying encrypted volumes is supported
You cannot change the
encryption state of a
volume with the modify
volume command
100. How much does it cost to modify a volume?
There is no cost for modifying the volume
You start to be charged for the
new volume specifications when
you issue the modify-volume
API call
101. What about permissions?
Modifying a volume’s type and PIOPS can dramatically
impact performance
We recommend you lockdown
permissions to the modify-
volume API like you would the
delete-volume API
106. Modify Volume Timeline
gp2
io1
Modify-volume API Modifying State
Completed State
Optimizing State
(seconds)
(performance between both volumes)
Target size available
(expand file system)
107. Test: Load During Modify Volume
Unmodified Volume:
Volume: New 100GiB gp2 volume
File System: ext4
Instance: m4.2xlarge
Load: fio 50/50 read/write to 10x 8GiB files
Modified Volume:
Volume: New 100GiB gp2 volume
File System: ext4
Instance: m4.2xlarge
Load: fio 50/50 read/write to 10x 8GiB files
Test: While the fio script was running, we modified the volume size to 200GiB
108. Test: Load During Modify Volume
Questions:
1) Which is the unmodified?
2) Which is the modified?
109. Test: Load During Modify Volume
Questions:
1) Which is the unmodified?
2) Which is the modified?
115. Infor
Infor provides business applications specialized by industry and built for
the cloud, with more than 65 million cloud users.
“We’re excited about the announcement of
the Elastic Volumes feature for Amazon
Elastic Block Store and are looking forward to
the flexibility it offers to easily right-size
systems,” said Jim Plourde, Vice President,
Infor Cloud Services. “The feature makes it
easy to move some of our colder data to sc1
to provide substantial savings on storage
costs.”
116. Acquia
"Elastic volumes make it possible to quickly and efficiently
scale EBS volumes for our services in an automated fashion
while avoiding mistakes and dataloss. This will enabled us to
maintain service uptime, reduce operational maintenance and
lower unused data costs while seamlessly scaling our
volumes on-demand as needed. “
117. HipChat
HipChat has a 45+ node elastic search
cluster, with 1TB 6k IOPS io1 volumes.
These were overprovisioned and they
were undertaking an effort to to scale
down to 3K IOPS/volume.
Benefit: With Elastic Volumes, HipChat was able to save 4-6 weeks of
engineering effort, yet still realized the benefit of a reduction in IOPS
spend. This cost reduction was achieved with next to no engineering
overhead, and no impact to customers.