Más contenido relacionado La actualidad más candente (20) Similar a Optimizing Costs as You Scale on AWS (ENT302) - AWS re:Invent 2018 (20) Más de Amazon Web Services (20) Optimizing Costs as You Scale on AWS (ENT302) - AWS re:Invent 20182. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Optimizing Costs as You Scale on AWS
Keith Jarrett
WW Cost Optimization Lead
Amazon Web Services
E N T 3 0 2
Megan Kennedy
Program Manager
Vanguard
Tim Treston
Cloud Economist
Vanguard
3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
You’re using AWS
and you like it!
4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
But maybe you are spending more
than you planned …
5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Or you’d just like to spend less
6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
With the AWS cloud,
you only pay for what
you use
7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How do you
ensure that you
only pay for what
you need?
8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What to expect
We will introduce our framework for optimizing
and managing your AWS spend by focusing on
paying only for what you need
Hear lessons learned directly from those who have
been successful building cost management and
optimization capabilities: Vanguard Investments
And leave you with actionable steps to take back
to the office to implement immediately
9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The cloud has changed the way
IT and engineering deliver value
11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Embracing cloud economics
“We’ve seen a 52% total cost of ownership reduction.”
Ben Wilson,CTO GE Oil & Gas
Regarding their 58% cost reduction,
“People who conclude the cloud is too expensive are
doing it wrong.” Jake Burns, VP of Cloud Services
“40% decrease in usage cost per hour,
while growing usage by 70%.”
12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
But it doesn’t just happen
13. The five pillars of cost optimization
Right-sizing your
instances
Pick the right
pricing model
Increase elasticity
Measuring &
monitoring
Match usage to
storage class
14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The cloud has changed the way
IT & finance operate
16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Traditional planning & procurement
The disruption of IT procurement
17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Traditional planning & procurement
The disruption of IT procurement
Disruption of procurement process
18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Re: Designing the process
19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cloud financial management framework
Cost
optimization
• Cost aware architecture
• Right-sizing & elasticity
• Storage optimization
• Reserved instances
• Spot usage
Measurement &
accountability
• Account strategy
• Tagging strategy &
maintenance
• Cost visibility & analysis
• Metrics tracking &
monitoring
Planning &
forecasting
• Business planning
• Usage forecasting
• Cost forecasting
• Budgeting
Cloud financial
operations
• Governance & controls
• Procurement strategy
• Automation
• Build your cloud team
• Tool adoption & training
• Establish culture of cost
accountability
20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Getting started measurement & accountability
21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Step 1: Establish your account strategy
22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Using AWS Organizations to automate account
creation and consolidated billing
Policy-based management for multiple AWS accounts
Control AWS service
use across accounts
Consolidate billing
and usage reporting
Automate account
creation
https://aws.amazon.com/blogs/security/how-to-use-aws-organizations-to-automate-end-to-end-account-creation/
23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Business unit (BU)
AWS Organizations: Account strategies
24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Organizations: Account strategies
Environment lifecycle
25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Organizations: Account strategies
Project-based
26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Organizations: Account strategies
Cost allocation tags
Environment
Project
Team
Application ID
Cost center
27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Organizations + budgets for monitoring
Alerts can provide
notifications when budget is
reached
Cost allocation tags
Environment
Project
Team
Application ID
Cost center
28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Step 2: Establish your tagging strategy
29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s in a tag?
30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Using tags to establish visibility & chargebacks
https://aws.amazon.com/answers/account-management/aws-tagging-strategies/
Stakeholders Examples
• Cost center
• Application or workload
• User
• Expiration date
• Automation support
31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Tools to help with missing or misspelled tags
2. AWS Config + Lambda1. AWS Tag Editor
https://bit.ly/co17_tag2
https://docs.aws.amazon.com/awsconsolehelpdocs/latest/gsg/scenario-implementing-tagging.html
32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Step 3: Set up cost reports in AWS Cost Explorer
33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Re: Introducing AWS Cost Explorer
AWS Cost Explorer lets you dive deeper into your
cost and usage data to identify trends, pinpoint
cost drivers, and detect anomalies
Get started quickly!
• Monthly and daily cost by service or linked
account
• Monthly EC2 running hours cost and usage
• Reserved instance reports
• RI utilization
• RI coverage
NOTE: All functionality is also available
programmatically via the AWS Cost Explorer API
34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cost and usage report
Group by: Instance type, linked account,
purchase option, region, service, plus
many more …
Select daily or monthly
view and time horizon
Spend (above—here shown grouped by
instance type) and usage (below)
Filters to remove data not
relevant to your analysis
And then export your analysis to CSV
Advanced options to do things
like include or exclude costs
related to credits, refunds,
support, and so on
All functionality is available programmatically via API
35. Using Cost Explorer to understand true cost
It’s an
accrual
world.
– J.R. Storment
Cloudability Co-Founder
Understanding “true” AWS spend:
• Blended (removes cost of RI hours)
• Unblended (shows how RIs were applied)
• Unblended amortized (smooths upfront RI payment)
36. It’s an
accrual
world.
– J.R. Storment
Cloudability Co-Founder
Upfront RI purchase
Non-amortized Amazon EC2 spend
Using Cost Explorer to understand true cost
37. It’s an
accrual
world.
– J.R. Storment
Cloudability Co-Founder
Upfront RI purchase
Non-amortized Amazon EC2 spend
Using Cost Explorer to understand true cost
38. Using Cost Explorer to understand true cost
It’s an
accrual
world.
– J.R. Storment
Cloudability Co-Founder
Upfront RI purchase
Non-amortized Amazon EC2 spend
Amortized Amazon EC2 spend
Amortized
payment
39. AWS Cost Explorer | RI Reporting
Reserved Instance (RI) Reporting
• [New! Reserved Instances dashboard in Cost
Explorer]
• RI savings-related information now available in
the RI utilization and coverage reports
• Availability of amortized RI cost data
• Supports Amazon EC2, Amazon RDS, Amazon
Redshift, Amazon ElastiCache, and Amazon
Elasticsearch reservations
NOTE: All functionality is also available
programmatically via the AWS Cost Explorer API
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
40. AWS Cost Explorer | RI purchase recommendations
Reserved Instance purchase recommendations
• Available to both payer and linked accounts
• RI purchase recommendations for the following
reservation models
• Amazon EC2
• Amazon RDS
• Amazon Redshift
• Amazon ElastiCache
• Amazon Elasticsearch
NOTE: All functionality is also available
programmatically via the AWS Cost Explorer API
41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Step 4: Leverage partners to accelerate optimization
42. Resource and cost management competency
Partner solutions designed to provide spend visibility, resource utilization, chargebacks, and workload costs
(Optimize Now)
https://aws.amazon.com/products/management-tools/partner-solutions/
43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Open invitation to join me
re:Invent 2019
44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Megan Kennedy
Program Manager
Vanguard
Tim Treston
Cloud Economist
Vanguard
45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What to expect
Introducing Vanguard
Cloud strategy & Cloud Business Office (CBO)
overview
Cloud financial management approach
46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Vanguard
One of the world’s largest investment management companies
19 16.5K
20+ M 170 $5.2T
47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CLIENTS
PROFITS
OWN OWN
VANGUARD
FUNDS
How we are different—operate at cost
Vanguard
48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
2016 2017 2018 2019…
Vanguard’s cloud journey
Public
cloud
MVP
web apps
MVP
analytics
Web apps
1.0
100% New
analytics apps
75%
Existing
web apps
100%
New
web apps
100%
Packages
Packages
1.0
100% Existing
web apps
Analytics
1.0
Analytics
2.0
100%
Existing
analytics
ENABLEMENTMIGRATION
MVP
packages
49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cloud Business Office (CBO)
CBO
Empowerment hub supporting cloud-based application development
Software delivery
lines of business
Information security
Legal
Risk
Compliance
Procurement
Cloud operations
Cloud engineering
Finance
Security, controls, risks
monitoring
Client engagement
Product management
50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cloud growth: The path we were on
0.5
1.0
1.5
2.0
2.5
MillionEC2hours
Actual EC2 hours Forecasted EC2 hours
EC2 usage growth
51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
0.5
1.0
1.5
2.0
2.5
MillionEC2hours
Actual EC2 hours Forecasted EC2 hours
EC2 usage growth and migration count
10
0
200
300
400
Migration count
Migrationcount
Cloud growth: The path we were on
52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
0.5
1.0
1.5
2.0
2.5
MillionEC2hours
Actual EC2 hours Forecasted EC2 hours
10
0
200
300
400
Migration count
Migrationcount
CBO
EC2 usage growth and migration count
Cloud growth: The path we were on
53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Phase I:
54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Phase I: React!
Purchasing
Governance
Education
55. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Phase I: Results
250
50
0
750
1,000
1,500
Dec-17
EC2 hours
EC2 usage growth and RI coverage
Nov-17Oct-17Sep-17Aug-17
1,250
ReservedOn-demand
EC2hours(000s)
33% RI coverage
56. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Phase II:
57. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Phase II: Manage growth
Governance
Education
Transparency
Purchasing
58. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
$0.20
$0.40
$0.80
$1.00
$1.40
May-17
LOB EC2 cost per hour
Apr-17Feb-17Dec-17Oct-17
$1.20
EC2costperhour
$0.60
Mar-17Jan-18Nov-17
Phase II: Manage growth
*46% decrease
in five months!
$1.20 / hr
$0.65 / hr
59. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Some good news stories …
• Have to look at the bill every day
instead of monthly
• We were having new conversations
and challenging SLAs
• Fundamental shift from a
provisioned to consumption model
$500
$1,000
$1,500
$2,000
$2,500
Amazon CloudWatch spend (daily)$3,000
Phase II: Results
60. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Some good news stories …
• Have to look at the bill every day
instead of monthly
• We were having new conversations
and challenging SLAs
• Fundamental shift from a
provisioned to consumption model
Line of business (LOB) ingest cluster hours
3500
3000
2500
2000
1500
1000
1-Jan 8-Jan 15-Jan 22-Jan 29-Jan 5-Feb 12-Feb
Phase II: Results
61. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Phase III:
62. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Phase III: Proactive management
Optimization
Governance
Accountability
Purchasing
63. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Phase III: Results
What did we learn?
• Accountability requires a strong
tagging policy
• Policy Enforcement
• Up-front automation
Support email tag
(% of assets tagged)
70%
90%
60%
50%
40%
80%
30%
20%
10%
64. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What did we learn?
• Implemented policy and
automation on sizing and run
schedules
• It pays to be proactive
0%
20%
40%
60%
80%
100%
0 20 40 60 80 100 120
%MemUsed
$Max CPU %
AUG—RDS utilization
Phase III: Results—RDS utilization (%)
65. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What did we learn?
• Implemented policy and
automation on sizing and run
schedules
• It pays to be proactive
0%
20%
40%
60%
80%
100%
0 20 40 60 80 100 120
%MemUsed Max CPU %
SEPT—RDS utilization
Phase III: Results—RDS utilization (%)
66. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
70%
90%
60%
50%
40%
80%
30%
20%
10%
100%
Phase III: Results—Coverage by payment option
Oct-17 Nov-17 Dec-17 Jan-18 Feb-18 Mar-18 Apr-18 May-18 Jun-18 Jul-18 Aug-18 Sep-18
ReservedOn-demand Spot
100% 100% 96% 97%
88%
64%
60%
37% 34% 37% 44% 43%
25%
35%
29%
25% 23%
31% 34% 31% 34%
10%
19% 20%
38%
3%4%
17% 20%
2%
Line of business usage by payment type
67. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lessons learned: What would we do differently?
• Tagging: Enforce tagging early and often—it makes everything easier
• RI purchases: Do the math and see what makes sense for you
• Set expectations: Enterprise discount programs and third-party tool
implementation take time—plan accordingly
• Be proactive: Iterate & automate
68. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Re: Designing the journey
• Cash view: OD • Cash view: OD + RI • Opex view: RI + OD
(amortization)
• Monthly cost data
• Align billing to org
• Central
infrastructure
• Manual reporting
• Show-back and
education
• LOB self service
• Charge-back
• LOB ownership
• Monthly approval RoB
• Train CBO
• Predefined approvals
• Cost workshops
• EDP coverage
• Cost opt tooling
• Reactive
• Manual shutdowns
• Conservative RI
Purchase
• Adv. RI purchase
• Automation
• Proactive
• Policy driven
automation & alerts
Phase 1 Phase 2 Phase 3
69. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
70. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Where to start
• Visit cost optimization resource center @
https://aws.amazon.com/pricing/cost-optimization/
• Enable cost explorer for your organization
• Leverage cost management partners
https://aws.amazon.com/products/management-tools/partner-
solutions/
• Join us this week to learn more!
71. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Want to learn more?
Wednesday, Nov 28
[WIN201] How to Optimize Your Microsoft Windows Workloads on AWS (ft. Xero)
11:30 a.m. – 12:30 p.m. | Venetian, Level 4, Delfino 4005
Thursday, Nov 29
[ENT 309] Building a World Class Cloud Center of Excellence (ft. Zendesk & CloudHealth)
3:15 p.m. – 4:15 p.m. | Venetian, Level 4, Delfino 4002
Tuesday, Nov 27
[ENT224] End Extra Spend: Hunting for Increased Value through Cost Optimization
3:15 p.m. – 5:30 p.m. | MGM, Level 3, Premier Ballroom 320
72. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Keith Jarrett
WW Cost Optimization Lead
Amazon Web Services
Megan Kennedy
Program
Manager
Vanguard
Tim Treston
Cloud Economist
Vanguard
73. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.