Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Velox at SF Data Mining Meetup

2.458 visualizaciones

Publicado el

Presentation on Velox from the UC Berkeley AMPLab at the 3/11 SF Data Mining Meetup (http://www.meetup.com/Data-Mining/events/220720883/).

Publicado en: Datos y análisis

Velox at SF Data Mining Meetup

  1. 1. VELOX: MODELS IN ACTION Dan Crankshaw UC Berkeley AMPLab crankshaw@cs.berkeley.edu Marin Software 2015
  2. 2. Algorithms, Machines, and People
  3. 3. Algorithms, Machines, and People “deep questions over dirty and heterogenous data”
  4. 4. Algorithms, Machines, and People “deep questions over dirty and heterogenous data”
  5. 5. Algorithms, Machines, and People “deep questions over dirty and heterogenous data”
  6. 6. Algorithms, Machines, and People “deep questions over dirty and heterogenous data”
  7. 7. GraphX Algorithms, Machines, and People “deep questions over dirty and heterogenous data”
  8. 8. GraphX Algorithms, Machines, and People “deep questions over dirty and heterogenous data”
  9. 9. GraphX Algorithms, Machines, and People “deep questions over dirty and heterogenous data”
  10. 10. BERKELEY DATA 
 ANALYTICS STACK (BDAS) Spark Spark Streaming Spark SQL BlinkDB GraphX MLlib MLBase HDFS, S3, … Tachyon Mesos HadoopYarn
  11. 11. Catify: Music for Cats
  12. 12. MODELINGTASK Rating Songs
  13. 13. MODELINGTASK Ratings Songs Prediction
  14. 14. Catify: Music for Cats
  15. 15. Catify: Music for Cats CatID Song Score 1 16 2.1 1 14 3.7 3 273 4.2 4 14 1.9
  16. 16. Catify: Music for Cats Pipeline CatID Song Score 1 16 2.1 1 14 3.7 3 273 4.2 4 14 1.9
  17. 17. Catify: Music for Cats Tachyon + HDFS Pipeline CatID Song Score 1 16 2.1 1 14 3.7 3 273 4.2 4 14 1.9
  18. 18. Catify: Music for Cats Tachyon + HDFS Pipeline CatID Song Score 1 16 2.1 1 14 3.7 3 273 4.2 4 14 1.9
  19. 19. Pipeline Tachyon + HDFS Node.js App Server Apache Web Server Catify: Music for Cats
  20. 20. Catify: Music for Cats Songs Users
  21. 21. Songs Users O(users * songs) Catify: Music for Cats
  22. 22. Pipeline Tachyon + HDFS Node.js App Server Apache Web Server Catify: Music for Cats
  23. 23. Pipeline Tachyon + HDFS Node.js App Server Apache Web Server Precomputed Ratings Catify: Music for Cats
  24. 24. Pipeline Tachyon + HDFS Node.js App Server Apache Web Server Precomputed Ratings Catify: Music for Cats Black box
  25. 25. Pipeline Tachyon + HDFS Node.js App Server Apache Web Server Training Data Precomputed Ratings Catify: Music for Cats Black box
  26. 26. Pipeline Tachyon + HDFS Node.js App Server Apache Web Server Training Data Precomputed Ratings Catify: Music for Cats Black box
  27. 27. What’s wrong?
  28. 28. 1. Serving system: low-latency but high staleness What’s wrong?
  29. 29. 1. Serving system: low-latency but high staleness 2. Batch training: slow incremental maintenance, no serving What’s wrong?
  30. 30. 1. Serving system: low-latency but high staleness 2. Batch training: slow incremental maintenance, no serving 3. Ad-hoc model management What’s wrong?
  31. 31. VELOX GOALS
  32. 32. VELOX GOALS 1. Low latency and fresh predictions
  33. 33. VELOX GOALS 1. Low latency and fresh predictions 2. Break the abstraction: model- specific optimizations
  34. 34. VELOX GOALS 1. Low latency and fresh predictions 2. Break the abstraction: model- specific optimizations 3. Unified system eases operation
  35. 35. Spark Spark Streaming Spark SQL BlinkDB GraphX MLlib MLBase HDFS, S3, … Tachyon THE MISSING PIECE IN BDAS Mesos
  36. 36. Spark Streaming Spark SQL Graph X ML library BlinkDB MLbase Training THE MISSING PIECE IN BDAS Spark HDFS, S3, … Tachyon Mesos
  37. 37. Spark Streaming Spark SQL Graph X ML library BlinkDB MLbase Training Management + Serving THE MISSING PIECE IN BDAS Spark HDFS, S3, … Tachyon Mesos
  38. 38. Spark Streaming Spark SQL Graph X ML library BlinkDB MLbase Velox Training Management + Serving THE MISSING PIECE IN BDAS Spark HDFS, S3, … Tachyon Mesos
  39. 39. Spark Streaming Spark SQL Graph X ML library BlinkDB MLbase Velox Training Management + Serving Spark HDFS, S3, … Tachyon THE MISSING PIECE IN BDAS Mesos
  40. 40. Spark Streaming Spark SQL Graph X ML library BlinkDB MLbase Velox Training Management + Serving Spark HDFS, S3, … Tachyon Model Manager THE MISSING PIECE IN BDAS Mesos
  41. 41. Spark Streaming Spark SQL Graph X ML library BlinkDB MLbase Velox Training Management + Serving Spark HDFS, S3, … Tachyon Model Manager Prediction Service THE MISSING PIECE IN BDAS Mesos
  42. 42. VELOX ARCHITECTURE
  43. 43. VELOX ARCHITECTURE Standalone Scala Service
  44. 44. VELOX ARCHITECTURE Standalone Scala Service Automatic Integration with Spark
  45. 45. VELOX ARCHITECTURE Standalone Scala Service Personalized Predictions as a Service Automatic Integration with Spark
  46. 46. VELOX ARCHITECTURE Standalone Scala Service Shared-Nothing Serving Cluster Personalized Predictions as a Service Automatic Integration with Spark
  47. 47. SYSTEM ARCHITECTURE uuid: 01-10 uuid: 11-20 uuid: 20-30
  48. 48. SYSTEM ARCHITECTURE frontend.js uuid: 01-10 uuid: 11-20 uuid: 20-30
  49. 49. SYSTEM ARCHITECTURE frontend.js uuid: 01-10 uuid: 11-20 uuid: 20-30 uuid: 4
  50. 50. SYSTEM ARCHITECTURE Predictions via REST frontend.js uuid: 01-10 uuid: 11-20 uuid: 20-30 uuid: 4
  51. 51. SYSTEM ARCHITECTURE Predictions via REST frontend.js Returns score uuid: 01-10 uuid: 11-20 uuid: 20-30 uuid: 4
  52. 52. SYSTEM ARCHITECTURE frontend.js uuid: 01-10 uuid: 11-20 uuid: 20-30 uuid: 4
  53. 53. SYSTEM ARCHITECTURE frontend.js uuid: 01-10 uuid: 11-20 uuid: 20-30 Feedback via REST uuid: 4
  54. 54. SYSTEM ARCHITECTURE frontend.js uuid: 01-10 uuid: 11-20 uuid: 20-30 Feedback via REST Model updated in realtime uuid: 4
  55. 55. SYSTEM ARCHITECTURE frontend.js uuid: 01-10 uuid: 11-20 uuid: 20-30
  56. 56. SYSTEM ARCHITECTURE master workerworker worker frontend.js uuid: 01-10 uuid: 11-20 uuid: 20-30
  57. 57. SYSTEM ARCHITECTURE master workerworker worker Batch train RPC frontend.js uuid: 01-10 uuid: 11-20 uuid: 20-30
  58. 58. SYSTEM ARCHITECTURE master workerworker worker Batch train RPC frontend.js uuid: 01-10 uuid: 11-20 uuid: 20-30 Returns batch trained model
  59. 59. Mesos Mesos HDFS, S3, … Tachyon HadoopYarn Spark Straming Shark SQL Graph X ML library BlinkDB MLbase Spark Velox Training Management + Serving PREDICTION SERVICE Model Manager Prediction Service
  60. 60. PREDICTION API GET  /velox/catify/predict?userid=22&song=27632 Simple point queries:
  61. 61. PREDICTION API GET  /velox/catify/predict_top_k?userid=22&k=100 GET  /velox/catify/predict?userid=22&song=27632 Simple point queries: More complex ordering queries:
  62. 62. PREDICTION API GET  /velox/catify/predict_top_k?userid=22&k=100 GET  /velox/catify/predict?userid=22&song=27632 Simple point queries: More complex ordering queries: Low-latency and scalable partitioning Personalized Predictions
  63. 63. PREDICTION API GET  /velox/catify/predict_top_k?userid=22&k=100 GET  /velox/catify/predict?userid=22&song=27632 Simple point queries: More complex ordering queries: Low-latency and scalable partitioning Personalized Predictions Intelligent Caching Sharing and re-use of model partial-state
  64. 64. SYSTEM ARCHITECTURE Predictions via REST frontend.js uuid: 01-10 uuid: 11-20 uuid: 20-30 uuid: 4
  65. 65. PREDICTION EXECUTION def  predict(  u:  UUID,  x:  Context  ) uuid model
  66. 66. PREDICTION EXECUTION def  predict(  u:  UUID,  x:  Context  ) uuid model Look up user model Read
  67. 67. PREDICTION EXECUTION def  predict(  u:  UUID,  x:  Context  ) uuid model Look up user model Primary key lookup Read
  68. 68. PREDICTION EXECUTION def  predict(  u:  UUID,  x:  Context  ) uuid model Look up user model Primary key lookup Partition queries by user: always local Read
  69. 69. Compute Features PREDICTION EXECUTION def  predict(  u:  UUID,  x:  Context  ) user independent }f( )
  70. 70. Compute Features PREDICTION EXECUTION def  predict(  u:  UUID,  x:  Context  ) Feature computation 
 could be costly user independent }f( )
  71. 71. Compute Features PREDICTION EXECUTION def  predict(  u:  UUID,  x:  Context  ) Feature computation 
 could be costly user independent } Cache features for reuse across users f( )
  72. 72. TOP-K QUERIES Query predicate to pre-filter candidate set All Songs
  73. 73. TOP-K QUERIES Query predicate to pre-filter candidate set All Songs Playlist Keywords
  74. 74. TOP-K QUERIES Query predicate to pre-filter candidate set All Songs Playlist Keywords Candidate Songs
  75. 75. TOP-K QUERIES Query predicate to pre-filter candidate set All Songs Playlist Keywords Candidate Songs Score and rank all candidates
  76. 76. TOP-K QUERIES Query predicate to pre-filter candidate set All Songs Playlist Keywords Candidate Songs By exploiting split model design we can leverage: Score and rank all candidates
  77. 77. TOP-K QUERIES Query predicate to pre-filter candidate set All Songs Playlist Keywords Candidate Songs By exploiting split model design we can leverage: Score and rank all candidates A. Shrivastava, P. Li. “Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS).” NIPS’14 Best Paper
  78. 78. TOP-K QUERIES Query predicate to pre-filter candidate set All Songs Playlist Keywords Candidate Songs By exploiting split model design we can leverage: Score and rank all candidates A. Shrivastava, P. Li. “Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS).” NIPS’14 Best Paper Y. Low and A. X. Zheng. “Fast Top-K Similarity Queries Via Matrix Compression.” CIKM 2012
  79. 79. SYSTEM ARCHITECTURE frontend.js uuid: 01-10 uuid: 11-20 uuid: 20-30 uuid: 4
  80. 80. SYSTEM ARCHITECTURE frontend.js Returns score uuid: 01-10 uuid: 11-20 uuid: 20-30 uuid: 4
  81. 81. Mesos Mesos HDFS, S3, … Tachyon HadoopYarn Spark Straming Shark SQL Graph X ML library BlinkDB MLbase Spark Velox Training Management + Serving Model Manager Prediction Service MODEL MANAGER
  82. 82. PERSONALIZED MODELING
  83. 83. PERSONALIZED MODELING
  84. 84. PERSONALIZED MODELING A Separate Model for Each User?
  85. 85. PERSONALIZED MODELING Computationally Inefficient many complex models A Separate Model for Each User?
  86. 86. PERSONALIZED MODELING Statistically Inefficient not enough data per user Computationally Inefficient many complex models A Separate Model for Each User?
  87. 87. Input (Song) Rating
  88. 88. Input (Song) Rating
  89. 89. Input (Song) Rating Split
  90. 90. Rating Split Input (Song)
  91. 91. PERSONALIZED SPLIT MODEL Input (Song)
  92. 92. PERSONALIZED SPLIT MODEL Input (Song) Shared Basis Feature Model
  93. 93. PERSONALIZED SPLIT MODEL Input (Song) Shared Basis Feature Model Big Data
  94. 94. PERSONALIZED SPLIT MODEL Input (Song) Shared Basis Feature Model Big Data Changes Slowly
  95. 95. PERSONALIZED SPLIT MODEL Input (Song) Shared Basis Feature Model Big Data Changes Slowly Train in Batch!
  96. 96. PERSONALIZED SPLIT MODEL Input (Song) Shared Basis Feature Model Big Data Changes Slowly Train in Batch! Personalized User Model
  97. 97. PERSONALIZED SPLIT MODEL Input (Song) Shared Basis Feature Model Big Data Changes Slowly Train in Batch! Small Data Personalized User Model
  98. 98. PERSONALIZED SPLIT MODEL Input (Song) Shared Basis Feature Model Big Data Changes Slowly Train in Batch! Small Data Changes Quickly Personalized User Model
  99. 99. PERSONALIZED SPLIT MODEL Input (Song) Shared Basis Feature Model Big Data Changes Slowly Train in Batch! Small Data Changes Quickly Train Online! Personalized User Model
  100. 100. Input (Song) Personalized User Model Shared Basis Feature Model PERSONALIZED SPLIT MODEL
  101. 101. Input (Song) Personalized User Model Input (Song) Shared Basis Feature Model PERSONALIZED SPLIT MODEL
  102. 102. Input (Song) Personalized User Model Input (Song) Shared Basis Feature Model PERSONALIZED SPLIT MODEL
  103. 103. Input (Song) Personalized User Model Meow Input (Song) Shared Basis Feature Model PERSONALIZED SPLIT MODEL
  104. 104. Personalized User Model Meow Input (Song) Input (Song) Shared Basis Feature Model PERSONALIZED SPLIT MODEL
  105. 105. Personalized User Model Meow Terrible Input (Song) Input (Song) Shared Basis Feature Model PERSONALIZED SPLIT MODEL
  106. 106. MATHEMATICAL FORMULATION Input (Song)
  107. 107. MATHEMATICAL FORMULATION Input (Song) x
  108. 108. Shared Basis Feature Models Changes slowly MATHEMATICAL FORMULATION Input (Song) x
  109. 109. Shared Basis Feature Models Changes slowly MATHEMATICAL FORMULATION Input (Song) f(x; ✓) x
  110. 110. Shared Basis Feature Models Personalized User Model Changes slowly Highly dynamic MATHEMATICAL FORMULATION Input (Song) f(x; ✓) x
  111. 111. Shared Basis Feature Models Personalized User Model Changes slowly Highly dynamic MATHEMATICAL FORMULATION Input (Song) f(x; ✓) · wu x
  112. 112. Shared Basis Feature Models Personalized User Model Changes slowly Highly dynamic = Rating MATHEMATICAL FORMULATION Input (Song) f(x; ✓) · wu x Meow
  113. 113. FEEDBACK API POST  /velox/catify/observe?userid=22&song=27&score=3.7 Simple direct value feedback:
  114. 114. FEEDBACK API POST  /velox/catify/observe?userid=22&song=27&score=3.7 Simple direct value feedback: Continuously update 
 user models inVelox Online Learning
  115. 115. FEEDBACK API POST  /velox/catify/observe?userid=22&song=27&score=3.7 Simple direct value feedback: Continuously update 
 user models inVelox Online Learning Offline Learning Logged toTachyon for feature learning in Spark
  116. 116. FEEDBACK API POST  /velox/catify/observe?userid=22&song=27&score=3.7 Simple direct value feedback: Continuously update 
 user models inVelox Online Learning Offline Learning Logged toTachyon for feature learning in Spark Evaluation Continuously assess
 model performance
  117. 117. SYSTEM ARCHITECTURE frontend.js uuid: 01-10 uuid: 11-20 uuid: 20-30 Feedback via REST Model updated in realtime uuid: 4
  118. 118. ONLINE LEARNING velox.jar user model def  observe(u:  UUID,  x:  Context,  y:  Score)
  119. 119. ONLINE LEARNING velox.jar user model def  observe(u:  UUID,  x:  Context,  y:  Score) Update user model with new training data Write
  120. 120. ONLINE LEARNING velox.jar user model def  observe(u:  UUID,  x:  Context,  y:  Score) Stochastic gradient descent Update user model with new training data Write
  121. 121. ONLINE LEARNING velox.jar user model def  observe(u:  UUID,  x:  Context,  y:  Score) Stochastic gradient descent Incremental linear algebra Update user model with new training data Write
  122. 122. SYSTEM ARCHITECTURE master workerworker worker Batch train RPC frontend.js uuid: 01-10 uuid: 11-20 uuid: 20-30
  123. 123. OFFLINE OR NEARLINE LEARNING def  retrain(trainingData:  RDD) Spark Based Training Algs.wu · f(x; ✓) Automated retraining policies Efficient batch training using Spark Incremental learning using Spark Streaming
  124. 124. Data Model
  125. 125. Data Model Sample Bias: model affects the training data.
  126. 126. ALWAYS SERVETHE BEST SONG? Songs Predicted Rating
  127. 127. ALWAYS SERVETHE BEST SONG? Songs Predicted Rating
  128. 128. VELOX SOLUTION Predicted Rating Songs With prob. 1- ϵ serve the best predicted song
  129. 129. VELOX SOLUTION Predicted Rating Songs With prob. 1- ϵ serve the best predicted song
  130. 130. Predicted Rating Songs With prob. 1- ϵ serve the best predicted song With prob. ϵ pick a random song VELOX SOLUTION
  131. 131. Predicted Rating Songs With prob. 1- ϵ serve the best predicted song With prob. ϵ pick a random song VELOX SOLUTION
  132. 132. Predicted Rating Songs With prob. 1- ϵ serve the best predicted song With prob. ϵ pick a random song Epsilon Greedy VELOX SOLUTION
  133. 133. Predicted Rating Songs With prob. 1- ϵ serve the best predicted song With prob. ϵ pick a random song Epsilon Greedy Active Learning Opportunity to explore new systems for this emerging analytics workload VELOX SOLUTION
  134. 134. BEYOND RECOMMENDER SYSTEMS
  135. 135. 1. Spam and anomaly detection BEYOND RECOMMENDER SYSTEMS
  136. 136. 1. Spam and anomaly detection 2. Device/location specific modeling BEYOND RECOMMENDER SYSTEMS
  137. 137. 1. Spam and anomaly detection 2. Device/location specific modeling 3. YOUR machine learning application BEYOND RECOMMENDER SYSTEMS
  138. 138. Spark Streaming Spark SQL Graph X ML library BlinkDB MLbase Velox Training Management + Serving Spark HDFS, S3, … Tachyon Model Manager Prediction Service THE MISSING PIECE IN BDAS Mesos
  139. 139. SUMMARY
  140. 140. Today: model training and serving relies on ad-hoc, manual processes spread across multiple systems SUMMARY
  141. 141. Today: model training and serving relies on ad-hoc, manual processes spread across multiple systems TheVelox system automatically maintains multiple models while providing low latency, fresh, and personalized predictions SUMMARY
  142. 142. Today: model training and serving relies on ad-hoc, manual processes spread across multiple systems TheVelox system automatically maintains multiple models while providing low latency, fresh, and personalized predictions Velox will be open-source: coming soon to BDAS SUMMARY
  143. 143. Today: model training and serving relies on ad-hoc, manual processes spread across multiple systems TheVelox system automatically maintains multiple models while providing low latency, fresh, and personalized predictions Velox will be open-source: coming soon to BDAS https://amplab.cs.berkeley.edu/projects/velox/ SUMMARY
  144. 144. Today: model training and serving relies on ad-hoc, manual processes spread across multiple systems TheVelox system automatically maintains multiple models while providing low latency, fresh, and personalized predictions Velox will be open-source: coming soon to BDAS https://amplab.cs.berkeley.edu/projects/velox/ crankshaw@cs.berkeley.edu SUMMARY
  145. 145. QUESTIONS?

×