Auto Scaling allows cloud resources to scale automatically in reaction to the dynamic needs of customers, which helps to improve application availability and reduce costs. New target tracking scaling policies for Auto Scaling make it easy to set up dynamic scaling for your application in just a few steps. With target tracking, you simply select a load metric for your application, set the target value, and Auto Scaling adjusts resources as needed to maintain that target. In this session, you will learn how you can use target tracking to setup sound scaling policies “without the fuss”, and improve availability under fluctuating demand. Netflix is spending $6 billion on original content this year, with shows like The Crown, Narcos, and Stranger Things, and plenty more in the future. Hear how they’re using target tracking scaling policies to improve performance, reliability and availability around the world at prime times, without over-provisioning - and without guesswork. They will share some best practices examples of how target tracking allows their infrastructure to automatically adapt to changing traffic patterns in order to keep their audience entertained and their costs on target.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Auto Scaling Prime Time
Target Tracking Hits the Bullseye at
Tara Van Unen – Product Marketing, AWS
Vadim Filanovsky – Performance and Reliability Engineering, Netflix
C M P 3 1 1
N o v e m b e r 3 0 , 2 0 1 7
AWS re:INVENT

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Auto Scaling group
ELBCPU utilization
Amazon EC2
instances
Dynamic scaling
D e c r e a s e c o s t s
I m p r o v e a v a i l a b i l i t y

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS customer survey 2016
…
Top Picks for Auto Scaling
MASTERING METRICS
MORE SERVICES
Speedy Set Up MORE SERVICES

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Multiple conditions
• Manual customization
• Granular control and tuning
Step scaling policies

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
T arg e t tracking
Set the target value on your
scaling metric and let target
tracking maintain your metric
T he rmostat
Set the temperature and let
the thermostat do the work
to maintain the temperature
Set it and forget it!
How it works
Cooling
74

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Speedy setup
Quick picks
• No conditions
• Self-optimizing
• Fastest scaling

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ELB
Amazon EC2
instances
Traffic
5
10
15
20
25
30
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Instances
CPU
Target Utilization CPU Utilization Instances
Traffic
Traffic
Scaling-out

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Application Load Balancer
Auto Scaling group
Amazon EC2 instances
Availability Zone B
Load Balancing Layer
Compute
Layer
Data
Layer
Amazon
DynamoDB
…Demo
Amazon EC2 instances
Availability Zone A

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster
Fleet
Aurora Replicas
EMR instance group
Scalable target Scalable dimension
Aurora
Service TasksECS
Spot Fleet request Spot InstancesEC2 Spot Fleet
AppStream 2.0
EMR
Core nodes
Task codes
AppStream Instances
Tables and global
secondary indices (GSI)
Provisioned capacityDynamoDB
More services

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vancouver, BC – The home of Auto Scaling team

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Seattle, WA – Amazon headquarters

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster footprintCluster footprint

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ASG size ASG throughput

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ASG size ASG throughput

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Chapter One
The need for Auto Scaling

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Encoding
556 Kbps 277 KbpsPerceptually optimal video encoding

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Encoding jobs run during off-peak
Normal usage
Encoding
3 am 7 pm

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Recommendations

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Regional failover
Other benefits of Auto Scaling
Red-black pushes Curing cancer
(Hack Day project)

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Auto
Scaling

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Chapter Two
Choosing the metric

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What metric?

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What metric, explained
Throughput per instance
Example: How much work I did
RequestCountPerTarget
Resource util. per instance
Example: How tired I am
avg. CPUUtilization
Pros: Direct measure of work; intuitive
Cons: Drifts over time
Pros: Requires less adjustment
Cons: More oscillation/jitter
VS

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Auto Scaling on multiple metrics
• Harder to reason about scaling behavior
• Different metrics might contradict each other, causing
oscillation
Typical Netflix setup
• Normal scale-up and -down on throughput
• Emergency scale-up on CPU (aka “the hammer rule”)

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Chapter Three
Setting the target

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What is my target?
• curl, ab, siege, nghttp, Jmeter, Gatling…

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Squeezing with live prod traffic
Proxy ASG
Server ASG
Squeeze ASG
Clone
Client ASGs
Normal traffic flow
Controlled
throughput

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Understanding failures
VS
TODO - graph here

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Chapter Four
Scaling policy setup

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What could go wrong?
70
90
110
Per instance throughput
Scale up
Scale down
0
100
200
300
400
0k
10k
20k
30k
40k
ASG size
ASG throughput
ASG
throughput
and size
Per-instance
throughput

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
“No rush” scaling
Problem: Scaling amounts
too small, cooldown too
long
Effect: Scaling lags behind
the traffic flow. Not
enough capacity at peak,
capacity wasted in trough.
Remedy: Increase scaling
amounts … or migrate to
target tracking!
0
100
200
300
400
15k
35k
55k
75k
ASG size
ASG throughput
40
80
120
160
Per instance
throughput
Scale up
Scale down

34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Twitchy scaling
Problem: Scale-up policy is
too aggressive
Effect: Unnecessary
capacity churn
Remedy: Reduce scale-up
amount, increase the
number of eval periods …
or migrate to target
tracking!
0
25
50
75
100
0k
2k
3k
5k
6k
ASG size
ASG throughput
40
60
80
100
Per instance
throughput
Scale up
Scale down

35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Should I stay or should I go?
Problem: -up and -down
thresholds are too close to
each other
Effect: Constant capacity
oscillation
Remedy: Move -up and -
down thresholds farther
apart … or migrate to
target tracking!
0
100
200
300
5k
15k
25k
35k
ASG size
ASG throughput
80
90
100
110
120
Per instance
throughput
Scale up
Scale down

36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Step vs target tracking setup
VS

37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Step vs target tracking setup
VS
and and
Scale-up
Scale-down

38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Chapter Five
Traffic patterns

39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Friday Saturday
What a difference
a day makes!
Understanding traffic

40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Zooming in on morning traffic
2k
6k
10k
14k
18k
1:30 2:30 3:30 4:30 5:30 6:30 7:30
Weekday vs weekend traffic
Friday Saturday
ASG is at the lowest size
+
Aggressive traffic ramp-up
Can Auto Scaling keep up?

41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
90
100
110
120
130
3:00 4:00 5:00 6:00 7:00
Per-instance throughput
30
60
90
120
150
3:00 4:00 5:00 6:00 7:00
ASG size
Too coldToo hot Saves from trouble Saves money
Target trackingStep vs
Target

42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS innovation Delivers Results
Target
tracking
Easier setup
Better demand
tracking
Per-second
billing
Hitting the bullseye
Productivity
Reliability
Cost

43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What are you scaling? #autoscaling
https://aws.amazon.com/autoscaling/getting-started

44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!