Amazon EC2 deepdive and a sprinkel of AWS Compute | AWS Floor28

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SSID: Guest
Password: Cube@11999
AMAZON EC2 DEEPDIVE  
AND A SPRINKLE OF AWS COMPUTE
Doron Rogov 
Technical Trainer

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Floor28 Agenda
GameDay
24 Oct
Big Data Day
14 Oct
Technical Sessions
Serverless Data Workshop
Big Data UG Meetup
ML & DL Day
15 Oct
Technical Sessions
SageMaker Workshop
ML&DL Meetup
DevOps Day
16 Oct
Technical Sessions
K8s Workshop
DevOps Meetup
DevOps Day
17 Oct
Technical Sessions
Spot Workshop
Databases Day
18 Oct
Technical Sessions
PyTorch Meetup
Builders Day
AppSync, Alexa & IoT
22 Oct
Technical Sessions
CDK Workshop
AWS IL UG Meetup
Technical Sessions
Serverless Workshop
Virtual assistants UG Meetup
Builders Day 
Serverless backend
21 Oct
Enterprise IT Day
23 Oct
Technical Sessions

In this session
•Understanding the factors that going into choosing an Amazon EC2 instance
•Defining system performance and how it is characterized for different workloads
•How Amazon EC2 instances deliver performance while providing flexibility and
agility
•How to make the most of your EC2 instance experience through the lens of several
instance types

Amazon Elastic Compute Cloud (EC2) is big
API
EC2
EC2
Instances
Networking
Purchase options

It all started with…
First launched in August 2006
M1 instance
“One size fits all” M1

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017
1
3 5 7
11 12 13
23
42
52
60
70
EC2 Instance Type Growth—Increased Choice

11 2012 2013 2014 2015 2016 2017
12 13
23
42
52
60
70
EC2 Instance Type Growth—Increased Choice
2018
Over 100 different instances

Amazon EC2 instances
Host server
Hypervisor
Guest 1 Guest 2 Guest nInstance

Instance generation
c5.xlargeInstance family
Instance size

Amazon EC2 Instance Types
General
Purpose
Compute
Optimized
Memory
Optimized
Accelerated
Computing
Storage
Optimized
t3 c5 / c5d r5 / r5d p3 h1
t2 c4 r4 p2 i3
m5 / m5d x1 / x1e g3 d2
m4 z1d f1

High Memory Instances
Instance Name Memory
Logical
Processors
Dedicated EBS
Bandwidth
Network
Bandwidth
u-6tb1.metal 6 TiB 448 14 Gbps 25 Gbps
24 TiB of memory coming in 2019

NUMA (…NUMA Hi)
• Non-uniform memory access
• Each processor in a multi-CPU system has local memory that
is accessible through a fast interconnect
• Each processor can also access memory from other CPUs,
but local memory access is a lot faster than remote memory
• Performance is related to the number of CPU sockets and
how they are connected—Intel QuickPath Interconnect (QPI)

r5.24xlarge
QPI
384GiB 384GiB
48 vCPUs 48 vCPUs

x1e.32xlarge
QPI
QPI
QPIQPI
QPI
976GiB
976GiB
976GiB
976GiB
32 vCPU’s 32 vCPU’s
32 vCPU’s 32 vCPU’s

Tip: Kernel support for NUMA balancing
• Linux Kernel 3.8+ and Windows Datacenter 2003+ support NUMA balancing
• Not always the most efficient
• Overhead of managing memory can slow down your app
• Does your app have more memory that fits in a single socket?
• Linux: set “numa=off” in grub to disable NUMA awareness
• Do you have many processes or a footprint less than a single socket?
• Linux: use “numactl” to restrict them to specific cores or nodes
• Example: numactl --cpunodebind=0 --membind=0 ./myapp.run
• Windows: use processor affinity to lock applications to specific cores

Instance sizing
c5.18xlarge
C72/R144
2 - c5.9xlarge
C8/R16
≈
4 - c5.4xlarge
≈
8 - c5.2xlarge
≈ ≈ etc≈etc

What’s a virtual CPU? (vCPU)
• A vCPU is typically a hyper-threaded physical core*
• On Linux, “A” threads enumerated before “B” threads
• On Windows, threads are interleaved
• Divide vCPU count by two to get core count
* Remember the “T” family is special

LSTOPO

Disable hyper-threading if you need to
• Useful for FPU heavy applications (e.g. Financial calculation, Engineering simulation)
• Use ‘lscpu’ to validate layout
• Hot offline the “B” threads
for cpunum in $(cat /sys/devices/system/cpu/cpu*/topology/
thread_siblings_list | cut -s -d, -f2- | tr ',' 'n' | sort -un); do
echo 0 | sudo tee /sys/devices/system/cpu/cpu${cpunum}/online
Done
• Set grub to only initialize the first half of all threads
maxcpus=20

lscpu - r5.24xlarge
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 96
On-line CPU(s) list: 0-95
Thread(s) per core: 2
Core(s) per socket: 24
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
Stepping: 4
CPU MHz: 1427.759
BogoMIPS: 5000.00
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 1024K
L3 cache: 33792K
NUMA node0 CPU(s): 0-23,48-71

Resource allocation
• All resources assigned to you are dedicated to your instance with no over-commitment*
• All vCPUs are dedicated to you
• Memory allocated is assigned only to your instance
• Network resources are partitioned to avoid “noisy neighbors”
• Curious about the number of instances per host? Use “dedicated hosts” as a guide.
*Again, the “T” family is special

P-state and C-state control
• c4.8xlarge, d2.8xlarge, m4.16xlarge, p2.16xlarge,
x1.16xlarge, f1.16xlarge, x1.32xlarge, i3.metal, etc
• By entering deeper idle states, non-idle cores can
achieve up to 300MHz higher clock frequencies
• But… deeper idle states require more time to exit,
may not be appropriate for latency-sensitive
workloads
• Linux: limit c-state by adding
“intel_idle.max_cstate=1” to grub
• Windows: no options to control c states

Tip: P-state control for AVX2
• If an application makes heavy use of AVX2 on all cores, the processor may attempt to draw
more power than it should
• Processor will transparently reduce frequency
• Frequent changes of CPU frequency can slow an application
sudo sh -c "echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo“ 
See also: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/processor_state_control.html

T3 instances
• Lowest cost Amazon EC2
instance at $0.0052 per hour
• Burstable performance
• Fixed allocation enforced with
CPU credits
• Unlimited mode by default
Model vCPU Baseline
CPU credits/
hour
Maximum
CPU credit
Memory (GiB)
t3.nano 2 5% 6 144 0.5
t3.micro 2 10% 12 288 1
t3.small 2 20% 24 576 2
t3.medium 2 20% 24 576 4
t3.large 2 30% 36 864 8
t3.xlarge 4 40% 96 2304 16
t3.2xlarge 8 40% 192 4608 32

How credits work
Baseline rate
Credit
balance
Burst rate
• A CPU credit provides the performance of a
full CPU core for one minute
• An instance earns CPU credits at a steady
rate
• An instance consumes credits when active
• Credits expire (leak) after 24 hours

Circa 2012

What is Nitro?
From the C5 launch:
Q. What is the new hypervisor for Amazon EC2?
A. The new hypervisor for Amazon EC2, introduced with the launch of C5 instances, is a
component that primarily provides CPU and memory isolation for C5 instances. VPC networking
and EBS storage resources are implemented by dedicated hardware components that are part of
all current generation EC2 instance families. It is built on core Linux Kernel-based Virtual Machine
(KVM) technology, but does not include general purpose operating system components.
• The Nitro Hypervisor is the “new hypervisor,” but more than just a hypervisor

Jan 2013 - CR1 (no Nitro)
Amazon 
RDS
IAM
Amazon
Linux
cr1.8xlarge
EBS Volumes
Hardware Software
DM
Instance Storage
DM DM DM
VPC Networking

Nov 2013 - C3 (early Nitro)
Amazon 
RDS
IAM
Amazon
Linux
c3.8xlarge
Enhanced Networking
Hardware Software
DMDM DM EBS Volumes
Instance Storage

Jan 2015 - C4
Amazon 
RDS
IAM
Amazon
Linux
c4.8xlarge
EBS Volumes
Enhanced Networking
Hardware Software
DM EBS Volumes

May 2016 - X1
Amazon 
RDS
IAM
Amazon
Linux
x1.32xlarge
Instance Storage
Enhanced Networking
Hardware Software
DM DM EBS Volumes
Instance Storage

Amazon 
RDS
IAM
Amazon
Linux
i3.16xlarge
EBS Volumes
Instance Storage
Enhanced Networking
Hardware Software
DM
DM EBS Volumes
I3 - Feb 2017

Nov 2017 - C5
Amazon 
RDS
IAM
c5.18xlarge
EBS Volumes
Enhanced Networking
Hardware Software
Nitro Hypervisor

Nov 2017 - EC2 Bare Metal
Amazon 
RDS
IAM
i3.metal
EBS Volumes
Instance Storage
Enhanced Networking
Hardware

The Nitro System
• Nitro Hypervisor
• Lightweight hypervisor
• Nitro Card
• Storage
• Networking
• Management
• Monitoring
• Security
• Nitro Security Chip
• Integrated into the motherboard

FAQs
Q: Will my existing AMIs work on Nitro-based instances?
Q: Will applications need to be modified?
Q: Will all new instance types be based on the Nitro System?
A: Yes. Most ENA capable AMIs have the necessary drivers.
A: Most of the time, no. Some applications have relied on undocumented behavior to
detect they are running within EC2 and they may require adjustment.
A: In the fullness of time, we expect most (if not all) new instance types to be Nitro-
based. We have no plans to convert existing instance types to Nitro and expect to
continue to launch Xen based instance types where appropriate.

P3 GPU Instances
• NVIDIA Tesla V100 Volta Based GPU
• 1 PetaFLOP of computational performance in a single instance
• Up to 2.6X and 14X performance improvement over P2 for High-Performance
Computing use-cases and Machine Learning use-cases (respectively)
Instance Size GPUs
Accelerator
(V100)
GPU Peer to
Peer
GPU Memory
(GB)
vCPUs
Memory
(GiB)
Network
Bandwidth
EBS
Bandwidth
P3.2xlarge 1 1 No 16 8 61 Up to 10Gbps 1.7Gbps
P3.8xlarge 4 4 NVLink 64 32 244 10Gbps 7Gbps
P3.16xlarge 8 8 NVLink 128 64 488 25Gbps 14Gbps

F1 FPGA instances
• Up to 8 Xilinx Virtex UltraScale Plus VU9p FPGAs in a single instance with four
high-speed DDR-4 per FPGA
• Largest size includes high performance FPGA interconnects via PCIe Gen3 (FPGA
Direct), and bidirectional ring (FPGA Link)
• Designed for hardware-accelerated applications including financial computing,
genomics, accelerated search, and image processing
Instance size FPGAs FPGA Link
FPGA
Direct
vCPUs
Memory
(GiB)
NVMe Network bandwidth
f1.2xlarge 1 - 8 122 1 x 470 Up to 10 Gigabit
f1.16xlarge 8 Y Y 64 976 4 x 940 25 Gbps

EBS performance
Instance
type
EBS-optimized
by default
Max EBS bandwidth
(Mbps)*
Expected EBS
throughput (MB/s)**
Max. IOPS  
(16 KB I/O size)**
Max network
bandwidth
r4.16xlarge Yes 14,000 1,750 75,000 25 Gb/s
r5d.24xlarge Yes 14,000 1,750 80,000 25 Gb/s
i3.16xlarge Yes 14,000 1,750 65,000 25 Gb/s
m5d.24xlarge Yes 14,000 1,750 80,000 25 Gb/s
x1.32xlarge Yes 10,000 1,250 65,000 10 Gb/s
• Instance size affects throughput
• Match your volume size and type to your instance
• RAID multiple EBS volumes together to achieve:

Summary: Getting the most out of EC2 instances
• Use latest generation instances when possible
• Monitor T3 CPU credits
• NUMA balancing
• C state and P state controls
• Enhanced networking
• Profile your application
But, before you go…

Architectural Evolution
Service Amazon EC2 Amazon ECS AWS Lambda
Unit VM Task Function
Level of
abstraction
H/W OS Runtime
How do I
choose?
I want to configure
servers, storage,
networking, and
my OS.
I want to run
servers, configure
applications, and
control scaling.
Run my code when
it’s needed.

“Launching new instances and running tests in
parallel is easy…[when choosing an instance] there is
no substitute for measuring the performance of your
full application.” 
-EC2 documentation

SSID: Guest
Password: Cube@11999
GAME DAY 
PUT YOUR SKILLS TO THE TEST 
OCT 24  
Register now: bit.ly/Floor28GameDay

Thank You!
Doron Rogov 
rogdoron@amazon.com 
in/doronrogov

Amazon EC2 deepdive and a sprinkel of AWS Compute | AWS Floor28

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Amazon EC2 deepdive and a sprinkel of AWS Compute | AWS Floor28

Similar a Amazon EC2 deepdive and a sprinkel of AWS Compute | AWS Floor28 (20)

Más de Amazon Web Services

Más de Amazon Web Services (20)

Amazon EC2 deepdive and a sprinkel of AWS Compute | AWS Floor28