SlideShare una empresa de Scribd logo
1 de 12
Descargar para leer sin conexión
in The PlanningSouth Bay SRE Meetup - 20160809
The Data
How many lines in our monthly AWS bill ?
QUIZ
A. 7 Million
B. 70 Million
C. 700 Million
D. 7 Billion
Tracking ~2,400 applications
Running on ~80 different instance types
Deployed in ~30 zones
Across ~60 accounts
How to inform and scale our capacity planning operations ?
-- GOALS --
provide context and transparency
focus on actionable insights
automate most repetitive tasks
Capacity Planning Analytics
Rohan
Sharma
Torio
Risianto
Sebastien
de Larquier
provide context and transparency
Build intuitive and interactive dashboards
e.g.: Devices Impact on Capacity
How many apps in the Netflix environment ?
A. 24
B. 240
C. 2,400
D. 24,000
QUIZ
Information at Scale
focus on actionable insights
e.g., Keeping an Eye on
Efficiency
For each metric, check:
➔ significant deviation
➔ change in trend
Link each card to a
detailed dashboard.
Cloud Capacity Analytics at Scale
Requirements:
❏ monitor and classify our current footprint
❏ forecast future capacity needs based on current usage and expected
events (hardware migrations, holidays, …)
❏ optimize on-demand and reserved capacity for reliability and cost
Picsou [piksu]
1. Scrooge McDuck in french.
2. Netflix’s comprehensive cloud
capacity analytics tool.
Cloud Capacity Planning at Scale
Cloud Capacity Planning at Scale
Requirements
❏ monitor and classify our current footprint
❏ forecast future capacity needs based on current usage and expected
events (hardware migrations, holidays, …)
❏ optimize on-demand and reserved capacity for reliability and cost
✔
Next 6-12 Months:
capacity alerts and forecast
recommendations for optimal reserved capacity
enforce recommendations
Thank You

Más contenido relacionado

La actualidad más candente

pp_nextgencloudcon 120915 FINAL v2
pp_nextgencloudcon 120915 FINAL v2pp_nextgencloudcon 120915 FINAL v2
pp_nextgencloudcon 120915 FINAL v2
Ajay Dankar
 
Lessons Learned - Monitoring the Data Pipeline at Hulu
Lessons Learned - Monitoring the Data Pipeline at HuluLessons Learned - Monitoring the Data Pipeline at Hulu
Lessons Learned - Monitoring the Data Pipeline at Hulu
DataWorks Summit
 
Running a Massively Parallel Self-serve Distributed Data System At Scale
Running a Massively Parallel Self-serve Distributed Data System At ScaleRunning a Massively Parallel Self-serve Distributed Data System At Scale
Running a Massively Parallel Self-serve Distributed Data System At Scale
Zhenzhong Xu
 
Михаил Максимов ( Software engineer, DataArt. AWS certified Solution Architect)
Михаил Максимов ( Software engineer, DataArt. AWS certified Solution Architect)Михаил Максимов ( Software engineer, DataArt. AWS certified Solution Architect)
Михаил Максимов ( Software engineer, DataArt. AWS certified Solution Architect)
DataArt
 

La actualidad más candente (20)

The Fermilab HEPCloud Facility
The Fermilab HEPCloud FacilityThe Fermilab HEPCloud Facility
The Fermilab HEPCloud Facility
 
Container world talk final 021716
Container world talk final 021716Container world talk final 021716
Container world talk final 021716
 
pp_nextgencloudcon 120915 FINAL v2
pp_nextgencloudcon 120915 FINAL v2pp_nextgencloudcon 120915 FINAL v2
pp_nextgencloudcon 120915 FINAL v2
 
Cloud world forum talk 062515
Cloud world forum talk 062515Cloud world forum talk 062515
Cloud world forum talk 062515
 
Lessons Learned - Monitoring the Data Pipeline at Hulu
Lessons Learned - Monitoring the Data Pipeline at HuluLessons Learned - Monitoring the Data Pipeline at Hulu
Lessons Learned - Monitoring the Data Pipeline at Hulu
 
Building a real-time, scalable and intelligent programmatic ad buying platform
Building a real-time, scalable and intelligent programmatic ad buying platformBuilding a real-time, scalable and intelligent programmatic ad buying platform
Building a real-time, scalable and intelligent programmatic ad buying platform
 
Extracting Value from IOT using Azure Cosmos DB, Azure Synapse Analytics and ...
Extracting Value from IOT using Azure Cosmos DB, Azure Synapse Analytics and ...Extracting Value from IOT using Azure Cosmos DB, Azure Synapse Analytics and ...
Extracting Value from IOT using Azure Cosmos DB, Azure Synapse Analytics and ...
 
Scaling Infrastructure at Picnic
Scaling Infrastructure at PicnicScaling Infrastructure at Picnic
Scaling Infrastructure at Picnic
 
Spotify's journey to GCP
Spotify's journey to GCPSpotify's journey to GCP
Spotify's journey to GCP
 
Running a Massively Parallel Self-serve Distributed Data System At Scale
Running a Massively Parallel Self-serve Distributed Data System At ScaleRunning a Massively Parallel Self-serve Distributed Data System At Scale
Running a Massively Parallel Self-serve Distributed Data System At Scale
 
PloneConf2017: serverless python for astronaut safety
PloneConf2017:  serverless python for astronaut safetyPloneConf2017:  serverless python for astronaut safety
PloneConf2017: serverless python for astronaut safety
 
Building the Serverless Container Experience: Kevin McGrath, Spotinst, Server...
Building the Serverless Container Experience: Kevin McGrath, Spotinst, Server...Building the Serverless Container Experience: Kevin McGrath, Spotinst, Server...
Building the Serverless Container Experience: Kevin McGrath, Spotinst, Server...
 
Azure event grid
Azure event gridAzure event grid
Azure event grid
 
Google Cloud and Data Pipeline Patterns
Google Cloud and Data Pipeline PatternsGoogle Cloud and Data Pipeline Patterns
Google Cloud and Data Pipeline Patterns
 
Serverless Architecture GCP In Production
Serverless Architecture GCP In ProductionServerless Architecture GCP In Production
Serverless Architecture GCP In Production
 
Михаил Максимов ( Software engineer, DataArt. AWS certified Solution Architect)
Михаил Максимов ( Software engineer, DataArt. AWS certified Solution Architect)Михаил Максимов ( Software engineer, DataArt. AWS certified Solution Architect)
Михаил Максимов ( Software engineer, DataArt. AWS certified Solution Architect)
 
Digital Transformation & Solvency II Simulations for L&G: Optimizing, Acceler...
Digital Transformation & Solvency II Simulations for L&G: Optimizing, Acceler...Digital Transformation & Solvency II Simulations for L&G: Optimizing, Acceler...
Digital Transformation & Solvency II Simulations for L&G: Optimizing, Acceler...
 
R, Spark, Tensorflow, H20.ai Applied to Streaming Analytics
R, Spark, Tensorflow, H20.ai Applied to Streaming AnalyticsR, Spark, Tensorflow, H20.ai Applied to Streaming Analytics
R, Spark, Tensorflow, H20.ai Applied to Streaming Analytics
 
Apache Cassandra at Videoplaza — Stockholm Cassandra Users — September 2013
Apache Cassandra at Videoplaza — Stockholm Cassandra Users — September 2013Apache Cassandra at Videoplaza — Stockholm Cassandra Users — September 2013
Apache Cassandra at Videoplaza — Stockholm Cassandra Users — September 2013
 
Kubernetes for Sales Engineers & Solutions Engineers–You Too Can Leverage Kub...
Kubernetes for Sales Engineers & Solutions Engineers–You Too Can Leverage Kub...Kubernetes for Sales Engineers & Solutions Engineers–You Too Can Leverage Kub...
Kubernetes for Sales Engineers & Solutions Engineers–You Too Can Leverage Kub...
 

Destacado

Yahoo! JAPANのサービス開発を10倍早くした社内PaaS構築の今とこれから
Yahoo! JAPANのサービス開発を10倍早くした社内PaaS構築の今とこれからYahoo! JAPANのサービス開発を10倍早くした社内PaaS構築の今とこれから
Yahoo! JAPANのサービス開発を10倍早くした社内PaaS構築の今とこれから
Yahoo!デベロッパーネットワーク
 
ヤフオク!の快適なカスタマー体験を支えるモバイルアプリのライブアップデート技術
ヤフオク!の快適なカスタマー体験を支えるモバイルアプリのライブアップデート技術ヤフオク!の快適なカスタマー体験を支えるモバイルアプリのライブアップデート技術
ヤフオク!の快適なカスタマー体験を支えるモバイルアプリのライブアップデート技術
Yahoo!デベロッパーネットワーク
 
Yahoo!ブラウザーアプリのプロダクトマネージャーが考えていること
Yahoo!ブラウザーアプリのプロダクトマネージャーが考えていることYahoo!ブラウザーアプリのプロダクトマネージャーが考えていること
Yahoo!ブラウザーアプリのプロダクトマネージャーが考えていること
Yahoo!デベロッパーネットワーク
 
行列ができるECサイトの悩み~ショッピングや決済の技術的問題と処方箋
行列ができるECサイトの悩み~ショッピングや決済の技術的問題と処方箋行列ができるECサイトの悩み~ショッピングや決済の技術的問題と処方箋
行列ができるECサイトの悩み~ショッピングや決済の技術的問題と処方箋
Yahoo!デベロッパーネットワーク
 

Destacado (13)

Yahoo! JAPAN の Ambari 活用事例 #yjdsnight
Yahoo! JAPAN の Ambari 活用事例 #yjdsnightYahoo! JAPAN の Ambari 活用事例 #yjdsnight
Yahoo! JAPAN の Ambari 活用事例 #yjdsnight
 
Netflix cloud architecture...continued
Netflix cloud architecture...continuedNetflix cloud architecture...continued
Netflix cloud architecture...continued
 
What an Enterprise Can Learn from Netflix, a Cloud-native Company (ENT203) | ...
What an Enterprise Can Learn from Netflix, a Cloud-native Company (ENT203) | ...What an Enterprise Can Learn from Netflix, a Cloud-native Company (ENT203) | ...
What an Enterprise Can Learn from Netflix, a Cloud-native Company (ENT203) | ...
 
Building out a Microservices Architecture with WebSphere Liberty Profile and ...
Building out a Microservices Architecture with WebSphere Liberty Profile and ...Building out a Microservices Architecture with WebSphere Liberty Profile and ...
Building out a Microservices Architecture with WebSphere Liberty Profile and ...
 
Yahoo! JAPANを支えるビッグデータプラットフォーム技術
Yahoo! JAPANを支えるビッグデータプラットフォーム技術Yahoo! JAPANを支えるビッグデータプラットフォーム技術
Yahoo! JAPANを支えるビッグデータプラットフォーム技術
 
Netflix Architecture Tutorial at Gluecon
Netflix Architecture Tutorial at GlueconNetflix Architecture Tutorial at Gluecon
Netflix Architecture Tutorial at Gluecon
 
Netflix Cloud Architecture and Open Source
Netflix Cloud Architecture and Open SourceNetflix Cloud Architecture and Open Source
Netflix Cloud Architecture and Open Source
 
OCP, Kubernetes ハイパースケールアーキテクチャ 導入の道のり - OpenStack最新情報セミナー(2016年7月)
OCP, Kubernetes  ハイパースケールアーキテクチャ 導入の道のり - OpenStack最新情報セミナー(2016年7月)OCP, Kubernetes  ハイパースケールアーキテクチャ 導入の道のり - OpenStack最新情報セミナー(2016年7月)
OCP, Kubernetes ハイパースケールアーキテクチャ 導入の道のり - OpenStack最新情報セミナー(2016年7月)
 
Netflix Global Cloud Architecture
Netflix Global Cloud ArchitectureNetflix Global Cloud Architecture
Netflix Global Cloud Architecture
 
Yahoo! JAPANのサービス開発を10倍早くした社内PaaS構築の今とこれから
Yahoo! JAPANのサービス開発を10倍早くした社内PaaS構築の今とこれからYahoo! JAPANのサービス開発を10倍早くした社内PaaS構築の今とこれから
Yahoo! JAPANのサービス開発を10倍早くした社内PaaS構築の今とこれから
 
ヤフオク!の快適なカスタマー体験を支えるモバイルアプリのライブアップデート技術
ヤフオク!の快適なカスタマー体験を支えるモバイルアプリのライブアップデート技術ヤフオク!の快適なカスタマー体験を支えるモバイルアプリのライブアップデート技術
ヤフオク!の快適なカスタマー体験を支えるモバイルアプリのライブアップデート技術
 
Yahoo!ブラウザーアプリのプロダクトマネージャーが考えていること
Yahoo!ブラウザーアプリのプロダクトマネージャーが考えていることYahoo!ブラウザーアプリのプロダクトマネージャーが考えていること
Yahoo!ブラウザーアプリのプロダクトマネージャーが考えていること
 
行列ができるECサイトの悩み~ショッピングや決済の技術的問題と処方箋
行列ができるECサイトの悩み~ショッピングや決済の技術的問題と処方箋行列ができるECサイトの悩み~ショッピングや決済の技術的問題と処方箋
行列ができるECサイトの悩み~ショッピングや決済の技術的問題と処方箋
 

Similar a Cloud Capacity Planning Tooling - South Bay SRE Meetup Aug-09-2016

Similar a Cloud Capacity Planning Tooling - South Bay SRE Meetup Aug-09-2016 (20)

Financial Services Analytics on AWS
Financial Services Analytics on AWSFinancial Services Analytics on AWS
Financial Services Analytics on AWS
 
AWS Summit Auckland Keynote
AWS Summit Auckland KeynoteAWS Summit Auckland Keynote
AWS Summit Auckland Keynote
 
Tooling Up for Efficiency: DIY Solutions @ Netflix - ABD319 - re:Invent 2017
Tooling Up for Efficiency: DIY Solutions @ Netflix - ABD319 - re:Invent 2017Tooling Up for Efficiency: DIY Solutions @ Netflix - ABD319 - re:Invent 2017
Tooling Up for Efficiency: DIY Solutions @ Netflix - ABD319 - re:Invent 2017
 
SaurabhKasyap
SaurabhKasyapSaurabhKasyap
SaurabhKasyap
 
WSO2 Analytics Platform - The one stop shop for all your data needs
WSO2 Analytics Platform - The one stop shop for all your data needsWSO2 Analytics Platform - The one stop shop for all your data needs
WSO2 Analytics Platform - The one stop shop for all your data needs
 
CA Performance Management 2.6 Deep Dive
CA Performance Management 2.6 Deep DiveCA Performance Management 2.6 Deep Dive
CA Performance Management 2.6 Deep Dive
 
Cyber Data Lake: How CIS Analyzes Billions of Network Traffic Records per Day
Cyber Data Lake: How CIS Analyzes Billions of Network Traffic Records per DayCyber Data Lake: How CIS Analyzes Billions of Network Traffic Records per Day
Cyber Data Lake: How CIS Analyzes Billions of Network Traffic Records per Day
 
Introduction to Azure Stream Analytics
Introduction to Azure Stream AnalyticsIntroduction to Azure Stream Analytics
Introduction to Azure Stream Analytics
 
Self-Tuning Data Centers
Self-Tuning Data CentersSelf-Tuning Data Centers
Self-Tuning Data Centers
 
CV
CVCV
CV
 
Les objets connectés : de nombreux cas d'usage
Les objets connectés : de nombreux cas d'usage Les objets connectés : de nombreux cas d'usage
Les objets connectés : de nombreux cas d'usage
 
The Next Wave of Computing
The Next Wave of ComputingThe Next Wave of Computing
The Next Wave of Computing
 
Netflix Edge Engineering Open House Presentations - June 9, 2016
Netflix Edge Engineering Open House Presentations - June 9, 2016Netflix Edge Engineering Open House Presentations - June 9, 2016
Netflix Edge Engineering Open House Presentations - June 9, 2016
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming Visualization
 
Abc inc
Abc incAbc inc
Abc inc
 
Supercharging Smart Meter BIG DATA Analytics with Microsoft Azure Cloud- SRP ...
Supercharging Smart Meter BIG DATA Analytics with Microsoft Azure Cloud- SRP ...Supercharging Smart Meter BIG DATA Analytics with Microsoft Azure Cloud- SRP ...
Supercharging Smart Meter BIG DATA Analytics with Microsoft Azure Cloud- SRP ...
 
How to Build Streaming Apps with Confluent II
How to Build Streaming Apps with Confluent IIHow to Build Streaming Apps with Confluent II
How to Build Streaming Apps with Confluent II
 
SQL In Hadoop: Big Data Innovation Without the Risk
SQL In Hadoop: Big Data Innovation Without the RiskSQL In Hadoop: Big Data Innovation Without the Risk
SQL In Hadoop: Big Data Innovation Without the Risk
 
Data-Driven Transformation: Leveraging Big Data at Showtime with Apache Spark
Data-Driven Transformation: Leveraging Big Data at Showtime with Apache SparkData-Driven Transformation: Leveraging Big Data at Showtime with Apache Spark
Data-Driven Transformation: Leveraging Big Data at Showtime with Apache Spark
 
MSBI_MSSQL_Bhrath
MSBI_MSSQL_BhrathMSBI_MSSQL_Bhrath
MSBI_MSSQL_Bhrath
 

Último

IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Enterprise Knowledge
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 

Último (20)

Evaluating the top large language models.pdf
Evaluating the top large language models.pdfEvaluating the top large language models.pdf
Evaluating the top large language models.pdf
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 

Cloud Capacity Planning Tooling - South Bay SRE Meetup Aug-09-2016

  • 1. in The PlanningSouth Bay SRE Meetup - 20160809 The Data
  • 2. How many lines in our monthly AWS bill ? QUIZ A. 7 Million B. 70 Million C. 700 Million D. 7 Billion
  • 3. Tracking ~2,400 applications Running on ~80 different instance types Deployed in ~30 zones Across ~60 accounts
  • 4. How to inform and scale our capacity planning operations ? -- GOALS -- provide context and transparency focus on actionable insights automate most repetitive tasks Capacity Planning Analytics Rohan Sharma Torio Risianto Sebastien de Larquier
  • 5. provide context and transparency Build intuitive and interactive dashboards
  • 6. e.g.: Devices Impact on Capacity
  • 7. How many apps in the Netflix environment ? A. 24 B. 240 C. 2,400 D. 24,000 QUIZ Information at Scale
  • 8. focus on actionable insights e.g., Keeping an Eye on Efficiency For each metric, check: ➔ significant deviation ➔ change in trend Link each card to a detailed dashboard.
  • 9. Cloud Capacity Analytics at Scale Requirements: ❏ monitor and classify our current footprint ❏ forecast future capacity needs based on current usage and expected events (hardware migrations, holidays, …) ❏ optimize on-demand and reserved capacity for reliability and cost Picsou [piksu] 1. Scrooge McDuck in french. 2. Netflix’s comprehensive cloud capacity analytics tool.
  • 11. Cloud Capacity Planning at Scale Requirements ❏ monitor and classify our current footprint ❏ forecast future capacity needs based on current usage and expected events (hardware migrations, holidays, …) ❏ optimize on-demand and reserved capacity for reliability and cost ✔ Next 6-12 Months: capacity alerts and forecast recommendations for optimal reserved capacity enforce recommendations