Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Techniques for Scaling the Netflix API - QCon SF

5.508 visualizaciones

Publicado el

This presentation was from QCon SF 2011. In these slides I discuss various techniques that we use to scale the API. I also discuss in more detail our effort around redesigning the API.

Publicado en: Tecnología, Empresariales

Techniques for Scaling the Netflix API - QCon SF

  1. 1. Techniques for Scaling the Netflix API By Daniel Jacobson @daniel_jacobson djacobson@netflix.com QCon SF 2011
  2. 2. Techniques for Scaling the Netflix API Agenda• History of the Netflix API• The Cloud• Development and Testing• Resiliency• Future of the Netflix API
  3. 3. Techniques for Scaling the Netflix API Agenda• History of the Netflix API• The Cloud• Development and Testing• Resiliency• Future of the Netflix API
  4. 4. Netflix API
  5. 5. Netflix API Requests by Audience At Launch In 2008 Netflix Devices Open API Developers
  6. 6. Netflix API
  7. 7. Netflix API Requests by Audience At Launch Today Netflix Devices Open API Developers
  8. 8. Public API Private API
  9. 9. Current Emphasis of Netflix API Netflix Devices
  10. 10. Techniques for Scaling the Netflix API Agenda• History of the Netflix API• The Cloud• Development and Testing• Resiliency• Future of the Netflix API
  11. 11. Discovery
  12. 12. Discovery
  13. 13. Streaming
  14. 14. Netflix API Powers Discovery
  15. 15. Netflix API : Requests Per Month 35 30 25Requests in Billions 20 15 10 5 -
  16. 16. Netflix API : Requests Per Month 35 30 25Requests in Billions 20 15 10 5 -
  17. 17. AWS Cloud
  18. 18. Autoscaling
  19. 19. Autoscaling
  20. 20. Techniques for Scaling the Netflix API Agenda• History of the Netflix API• The Cloud• Development and Testing• Resiliency• Future of the Netflix API
  21. 21. Development / Testing Philosophy Act fast, react fast
  22. 22. That Doesn’t Mean We Don’t Test• Unit tests• Functional tests• Regression scripts• Continuous integration• Capacity planning• Load / Performance tests
  23. 23. Development Contiuous Run Unit Tests(Feature & Test) Integration Perform Deploy to Run Functional Integration Staging Env Tests Tests Deploy to Customers
  24. 24. Cloud-Based Deployment Techniques
  25. 25. API Requests from the Internet Problem!Current CodeIn Production Single Canary Instance To Test New Code with Production Traffic (typically around 1-5% of traffic)
  26. 26. API Requests from the InternetCurrent CodeIn Production
  27. 27. API Requests from the InternetCurrent Code New CodeIn Production Getting Prepared for Production
  28. 28. API Requests from the Internet Old Code Current CodePrepared For Rollback In Production
  29. 29. API Requests from the Internet Old Code New CodeRolled Back into Production Out of Production
  30. 30. API Requests from the InternetCurrent CodeIn Production
  31. 31. API Requests from the InternetCurrent Code New CodeIn Production Getting Prepared for Production
  32. 32. API Requests from the Internet Old Code Current CodePrepared For Rollback In Production
  33. 33. API Requests from the Internet Current Code In Production
  34. 34. Development Continuous Run Unit Tests(Feature & Test) Integration Run Deploy to Run Functional Integration Staging Env Tests Tests PerformDeploy Canary Canary Instance(s) Analysis Deploy to Perform Black Deploy Black Customers Analysis Instances
  35. 35. Techniques for Scaling the Netflix API Agenda• History of the Netflix API• The Cloud• Development and Testing• Resiliency• Future of the Netflix API
  36. 36. APIPersonaliz Movie Movie Similar A/B Test ation User Info Reviews Engine Metadata Ratings Movies Engine
  37. 37. APIPersonaliz Movie Movie Similar A/B Test ation User Info Reviews Engine Metadata Ratings Movies Engine
  38. 38. APIPersonaliz Movie Movie Similar A/B Test ation User Info Reviews Engine Metadata Ratings Movies Engine
  39. 39. APIPersonaliz Movie Movie Similar A/B Test ation User Info Reviews Engine Metadata Ratings Movies Engine
  40. 40. Circuit Breaker Dashboard
  41. 41. Call Volume and Health / Last 10 Seconds
  42. 42. Call Volume / Last 2 Minutes
  43. 43. Successful Requests
  44. 44. Successful, But Slower Than Expected
  45. 45. Short-Circuited Requests, Delivering Fallbacks
  46. 46. Timeouts, Delivering Fallbacks
  47. 47. Thread Pool & Task Queue Full, Delivering Fallbacks
  48. 48. Exceptions, Delivering Fallbacks
  49. 49. # + # + # + # / (# + # + # + # + #) = Error Rate Error Rate
  50. 50. Status of Fallback Circuit
  51. 51. Requests per Second, Over Last 10 Seconds
  52. 52. SLA Information
  53. 53. APIPersonaliz Movie Movie Similar A/B Test ation User Info Reviews Engine Metadata Ratings Movies Engine
  54. 54. APIPersonaliz Movie Movie Similar A/B Test ation User Info Reviews Engine Metadata Ratings Movies Engine
  55. 55. APIPersonaliz Movie Movie Similar A/B Test ation User Info Reviews Engine Metadata Ratings Movies Engine
  56. 56. API FallbackPersonaliz Movie Movie Similar A/B Test ation User Info Reviews Engine Metadata Ratings Movies Engine
  57. 57. API FallbackPersonaliz Movie Movie Similar A/B Test ation User Info Reviews Engine Metadata Ratings Movies Engine
  58. 58. Techniques for Scaling the Netflix API Agenda• History of the Netflix API• The Cloud• Development and Testing• Resiliency• Future of the Netflix API
  59. 59. Netflix API
  60. 60. Netflix API Requests by Audience Supporting Streaming Devices Today Netflix Devices Open API Developers
  61. 61. Redesign the API
  62. 62. Netflix API : Requests Per Month 35 30 25Requests in Billions 20 15 10 5 -
  63. 63. Growth of the Netflix APIOver 30 Billion requests per month (Peaks at about 20,000 requests per second)
  64. 64. <catalog_titles> <number_of_results>1140</number_of_results> <start_index>0</start_index> <results_per_page>10</results_per_page> <catalog_title> <id>http://api.netflix.com/catalog/titles/movies/60021896</id><title short="Star" regular="Star"></title> <box_art small="http://alien2.netflix.com/us/boxshots/tiny/60021896.jpg" medium="http://alien2.netflix.com/us/boxshots/small/60021896.jpg" large="http://alien2.netflix.com/us/boxshots/large/60021896.jpg"></box_art> <link href="http://api.netflix.com/catalog/titles/movies/60021896/synopsis" rel="http://schemas.netflix.com/catalog/titles/synopsis" title="synopsis"></link> <release_year>2001</release_year> <category scheme="http://api.netflix.com/catalog/titles/mpaa_ratings" label="NR"></category> <category scheme="http://api.netflix.com/categories/genres" label="Foreign"></category> <link href="http://api.netflix.com/catalog/titles/movies/60021896/cast" rel="http://schemas.netflix.com/catalog/people.cast" title="cast"></link><link href="http://api.netflix.com/catalog/titles/movies/60021896/screen_formats" rel="http://schemas.netflix.com/catalog/titles/screen_formats" title="screenformats"></link <link href="http://api.netflix.com/catalog/titles/movies/60021896/languages_and_audio" rel="http://schemas.netflix.com/catalog/titles/languages_and_audio"title="languages and audio"></link> <average_rating>1.9</average_rating> <link href="http://api.netflix.com/catalog/titles/movies/60021896/similars" rel="http://schemas.netflix.com/catalog/titles.similars" title="similars"></link> <link href="http://www.netflix.com/Movie/Star/60021896" rel="alternate" title="webpage"></link> </catalog_title> <catalog_title> <id>http://api.netflix.com/catalog/titles/movies/17985448</id><title short="Lone Star" regular="Lone Star"></title> <box_art small="http://alien2.netflix.com/us/boxshots/tiny/17985448.jpg" medium="http://alien2.netflix.com/us/boxshots/small/17985448.jpg" large=""></box_art> <link href="http://api.netflix.com/catalog/titles/movies/17985448/synopsis" rel="http://schemas.netflix.com/catalog/titles/synopsis" title="synopsis"></link> <release_year>1996</release_year> <category scheme="http://api.netflix.com/catalog/titles/mpaa_ratings" label="R"></category> <category scheme="http://api.netflix.com/categories/genres" label="Drama"></category><link href="http://api.netflix.com/catalog/titles/movies/17985448/awards" rel="http://schemas.netflix.com/catalog/titles/awards" title="awards"></link> <link href="http://api.netflix.com/catalog/titles/movies/17985448/format_availability" rel="http://schemas.netflix.com/catalog/titles/format_availability"title="formats"></link> <link href="http://api.netflix.com/catalog/titles/movies/17985448/screen_formats" rel="http://schemas.netflix.com/catalog/titles/screen_formats" title="screenformats"></link> <link href="http://api.netflix.com/catalog/titles/movies/17985448/languages_and_audio" rel="http://schemas.netflix.com/catalog/titles/languages_and_audio"title="languages and audio"></link> <average_rating>3.7</average_rating> <link href="http://api.netflix.com/catalog/titles/movies/17985448/previews" rel="http://schemas.netflix.com/catalog/titles/previews" title="previews"></link> <link href="http://api.netflix.com/catalog/titles/movies/17985448/similars" rel="http://schemas.netflix.com/catalog/titles.similars" title="similars"></link> <link href="http://www.netflix.com/Movie/Lone_Star/17985448" rel="alternate" title="webpage"></link> </catalog_title></catalog_titles>
  65. 65. {"catalog_title":{"id":"http://api.netflix.com/catalog/titles/movies/60034967","title":{"title_short":"Rosencrantz and Guildenstern Are Dead","regular":"Rosencrantz and Guildenstern Are Dead"},"maturity_level":60,"release_year":"1990","average_rating":3.7,"box_art":{"284pix_w":"http://cdn-7.nflximg.com/en_US/boxshots/ghd/60034967.jpg","110pix_w":"http://cdn-7.nflximg.com/en_US/boxshots/large/60034967.jpg","38pix_w":"http://cdn-7.nflximg.com/en_US/boxshots/tiny/60034967.jpg","64pix_w":"http://cdn-7.nflximg.com/en_US/boxshots/small/60034967.jpg","150pix_w":"http://cdn-7.nflximg.com/en_US/boxshots/150/60034967.jpg","88pix_w":"http://cdn-7.nflximg.com/en_US/boxshots/88/60034967.jpg","124pix_w":"http://cdn-7.nflximg.com/en_US/boxshots/124/60034967.jpg"},"language":"en","web_page":"http://www.netflix.com/Movie/Rosencrantz_and_Guildenstern_Are_Dead/60034967","tiny_url":"http://movi.es/ApUP9"},"meta":{"expand":["@directors","@bonus_materials","@cast","@awards","@short_synopsis","@synopsis","@box_art","@screen_formats","@"links":{"id":"http://api.netflix.com/catalog/titles/movies/60034967","languages_and_audio":"http://api.netflix.com/catalog/titles/movies/60034967/languages_and_audio","title":"http://api.netflix.com/catalog/titles/movies/60034967/title","screen_formats":"http://api.netflix.com/catalog/titles/movies/60034967/screen_formats","cast":"http://api.netflix.com/catalog/titles/movies/60034967/cast","awards":"http://api.netflix.com/catalog/titles/movies/60034967/awards","short_synopsis":"http://api.netflix.com/catalog/titles/movies/60034967/short_synopsis","box_art":"http://api.netflix.com/catalog/titles/movies/60034967/box_art","synopsis":"http://api.netflix.com/catalog/titles/movies/60034967/synopsis","directors":"http://api.netflix.com/catalog/titles/movies/60034967/directors","similars":"http://api.netflix.com/catalog/titles/movies/60034967/similars","format_availability":"http://api.netflix.com/catalog/titles/movies/60034967/format_availability"}}}
  66. 66. Improve Efficiency of API RequestsCould it have been 5 billion requests per month? Or less? (Assuming everything else remained the same)
  67. 67. Netflix API : Requests Per Month 35 30 25Request in Billions 20 15 10 5 0
  68. 68. Netflix API : Requests Per Month 35 30 25Request in Billions 20 15 10 5 0
  69. 69. API Billionaires Club13 billion API calls / day (May 2011)Over 260 billion objects stored in S3 (January 2011)5 billion API calls / day (April 2010)5 billion API calls / day (October 2009)1 billion API calls / day (October 2011)8 billion API calls / month (Q3 2009)3.2 billion API-delivered stories / month (October2011)3 billion API calls / month (March 2009) Courtesy of John Musser, ProgrammableWeb
  70. 70. API Billionaires Club13 billion API calls / day (May 2011)Over 260 billion objects stored in S3 (January 2011)5 billion API calls / day (April 2010)5 billion API calls / day (October 2009)1 billion API calls / day (October 2011)8 billion API calls / month (Q3 2009)3.2 billion API-delivered stories / month (October2011)3 billion API calls / month (March 2009) Courtesy of John Musser, ProgrammableWeb
  71. 71. Two Major Objective for API Redesign• Improve performance for devices – Minimize network traffic (one API call, if possible) – Only deliver bytes that are needed• Improve ability for device teams to rapidly innovate – Customized request/response patterns per device – Deployment is independent from API schedules
  72. 72. Current Client / Server InteractionCLIENT APPS API SERVER AUTH SETUP TIME QUEUE LISTS LIST ( i ) TITLES
  73. 73. Future Client / Server InteractionCLIENT APPS API SERVER CUSTOM SCRIPTING TIER GENERIC API
  74. 74. Custom Scripting Tier InteractionCLIENT APPS API SERVER AUTH CUSTOM SCRIPTING SETUP TIER TIME PS3 HOME QUEUE SCREEN CUSTOM LISTS ENDPOINT LIST ( i ) Owned and operated by the UI teams TITLES
  75. 75. For More Titles From a List
  76. 76. Generic API InteractionCLIENT APP API SERVER AUTH GENERIC CONFIG SCRIPT TIME SIMPLE QUEUE CATALOG REQUEST TO API LISTS LIST ( i ) Owned and operated by the API team TITLES
  77. 77. Technologies AUTH CONFIG TIME GROOVY SIMPLE CLIENT COMPILED CATALOG JAVAQUEUE REQUESTLANGUAGE INTO TO API LOLOMO LISTS JVM LIST ( i ) TITLES
  78. 78. API Production Servers Cassandra Cluster UI Engineers
  79. 79. Active 1 2 3 4 5 SCRIPT REPOSITORY 6 (DYNAMICDEPLOYMENT) API SERVER APPLICATION CODE (REQUIRES API CODE PUSHES)
  80. 80. Active 1 2 3 4 5 SCRIPT REPOSITORY 6 (DYNAMIC 7DEPLOYMENT) API SERVER APPLICATION CODE (REQUIRES API CODE PUSHES)
  81. 81. 1 2 3 4 Active 5 SCRIPT REPOSITORY 6 (DYNAMIC 7DEPLOYMENT) API SERVER APPLICATION CODE (REQUIRES API CODE PUSHES)
  82. 82. If you are interested in helping us solve these problems, please contact me at: Daniel Jacobson djacobson@netflix.com @daniel_jacobsonhttp://www.linkedin.com/in/danieljacobson http://www.slideshare.net/danieljacobson

×