Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
Próxima SlideShare
Cargando en…5
×

# Introduction to Machine Learning for Taxify/Bolt

152 visualizaciones

Lecture slides that explain basics of Machine Learning using Linear Regression and Decision Tree algorithms as examples.

• Full Name
Comment goes here.

Are you sure you want to Yes No
• Sé el primero en comentar

### Introduction to Machine Learning for Taxify/Bolt

1. 1. INTRO TO MACHINE LEARNING 150 MIN 5.0 DMYTRO FISHMAN UNIVERSITY OF TARTU INSTITUTE OF COMPUTER SCIENCE
2. 2. New York City Taxi Fare Prediction https://www.kaggle.com/c/new-york-city-taxi-fare-prediction
3. 3. x y -0.8 0.2 -0.6 -0.4 -0.2 0.0 0.4 0.6 -0.75 -0.50 -0.25 0.00 0.25 0.50 0.75 1.00 type in your browser: tinyurl.com/yxb5k5jl (save a copy to your drive)
4. 4. The following slides are inspired by “An Introduction to Linear Regression Analysis” video https://youtu.be/zPG4NjIkCjc
5. 5. y X independent variable dependentvariable Linear Regression
6. 6. y X independent variable dependentvariable Linear Regression How the change in independent variable inﬂuences dependent variable?
7. 7. y X independent variable dependentvariable Positive relationship Linear Regression
8. 8. y X independent variable dependentvariable Negative relationship Linear Regression
9. 9. y X independent variable dependentvariable Linear Regression In order to build a linear regression we need observations
10. 10. y X independent variable dependentvariable In order to build a linear regression we need observations Linear Regression
11. 11. y X independent variable dependentvariable Linear Regression
12. 12. y X independent variable dependentvariable We want to ﬁnd a line such that … Linear Regression
13. 13. y X independent variable dependentvariable We want to ﬁnd a line such that … … it minimises the sum of errors Linear Regression
14. 14. y X independent variable dependentvariable actual estimated error We want to ﬁnd a line such that … … it minimises the sum of errors Linear Regression
15. 15. y X independent variable dependentvariable arg min = n ∑ i=1 ( − )2yi ̂yi Regression Line Least squares method We want to ﬁnd a line such that … … it minimises the sum of errors Linear Regression
16. 16. y X independent variable dependentvariable Linear Regression
17. 17. y X fareamount distance Linear Regression
18. 18. y X fareamount ̂y xw0 w1+= distance Linear Regression
19. 19. y X fareamount xw0 w1+= arg min , = n ∑ i=1 ( − )2yi ̂yi w0 w1 distance ̂y Linear Regression minimises the sum of errors with respect to w0 and w1w0 w1
20. 20. y X fareamount Linear Regression (example) distance 2 3 4 5 6 1 1 2 3 4 5 x y x - x̄ y - ȳ (x - x̄ )2 (x - x̄ )(y - ȳ) 1 2 -2 -2 4 4 2 4 -1 0 1 0 3 5 0 1 0 0 4 4 1 0 1 0 5 5 2 1 4 2 x̄ = 3 ȳ = 4 10 6 xw0 w1+=̂y w1 3w0 .6+=4 * w0 = 2.2 2.2 = ∑ (x − x)(y − y) ∑ (x − x)2 = 6 10 = .6
21. 21. = ∑ (x − x)(y − y) ∑ (x − x)2 = 6 10 = .6 y X fareamount Linear Regression (example) distance 2 3 4 5 6 1 1 2 3 4 5 x y x - x̄ y - ȳ (x - x̄ )2 (x - x̄ )(y - ȳ) 1 2 -2 -2 4 4 2 4 -1 0 1 0 3 5 0 1 0 0 4 4 1 0 1 0 5 5 2 1 4 2 x̄ = 3 ȳ = 4 10 6 xw0 w1+=̂y w1 3w0 .6+=4 * w0 = 2.2 2.2 Let’s return to our Colabs
22. 22. Decision Tree Algorithm By asking a simple question about value of independent variable it tries to predict a value of dependent variable
23. 23. Decision Tree Algorithm By asking a simple question about value of independent variable it tries to predict a value of dependent variable y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5
24. 24. Decision Tree Algorithm By asking a simple question about value of independent variable it tries to predict a value of dependent variable Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5
25. 25. Decision Tree Algorithm By asking a simple question about value of independent variable it tries to predict a value of dependent variable Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y False
26. 26. Decision Tree Algorithm By asking a simple question about value of independent variable it tries to predict a value of dependent variable Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True
27. 27. Decision Tree Algorithm By asking a simple question about value of independent variable it tries to predict a value of dependent variable Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True Root node
28. 28. Decision Tree Algorithm By asking a simple question about value of independent variable it tries to predict a value of dependent variable Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True Root node Left child Right child
29. 29. Decision Tree Algorithm By asking a simple question about value of independent variable it tries to predict a value of dependent variable Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True Root node Left child Right child Leafs
30. 30. Decision Tree Algorithm By asking a simple question about value of independent variable it tries to predict a value of dependent variable Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True
31. 31. Decision Tree Algorithm Here, X may correspond to any vertical line. Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True For example if X = 2.5: 2.5
32. 32. Decision Tree Algorithm Here, X may correspond to any vertical line. Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True For example if X = 2.5: 2.5 What are most reasonable values for Y and Z?
33. 33. Decision Tree Algorithm Here, X may correspond to any vertical line. Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True For example if X = 2.5: 2.5 What are most reasonable values for Y and Z (that minimise total MSE)?
34. 34. Decision Tree Algorithm What would be MSE if Y = 4 and Z = 5? Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 4 fare amount = 5 False True For example if X = 2.5: 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 4 Z = 5
35. 35. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 4 fare amount = 5 False True 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 4 Z = 5 yi ̂yiMSE = 1 n n ∑ i=1 ( − )2
36. 36. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 4 fare amount = 5 False True 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 4 Z = 5 yi ̂yiMSE = 1 n n ∑ i=1 ( − )2 real value predicted value
37. 37. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 4 fare amount = 5 False True 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 4 Z = 5 MSE = 1 n n ∑ i=1 ( − )2 = (y1 − ̂y1)2 + (y2 − ̂y2)2 + (y3 − ̂y3)2 + (y4 − ̂y4)2 + (y5 − ̂y5)2 5 yi ̂yi 1 2 3 4 5
38. 38. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 4 fare amount = 5 False True 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 4 Z = 5 MSE = 1 n n ∑ i=1 ( − )2 = (y1 − ̂y1)2 + (y2 − ̂y2)2 + (y3 − ̂y3)2 + (y4 − ̂y4)2 + (y5 − ̂y5)2 5 yi ̂yi 1 2 3 4 5
39. 39. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 4 fare amount = 5 False True 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 4 Z = 5 MSE = 1 n n ∑ i=1 ( − )2 = (2)2 + (0)2 + (0)2 + (1)2 + (0)2 5 yi ̂yi 1 2 3 4 5
40. 40. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 4 fare amount = 5 False True 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 4 Z = 5 MSE = 1 n n ∑ i=1 ( − )2 = 4 + 0 + 0 + 1 + 0 5 yi ̂yi 1 2 3 4 5
41. 41. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 4 fare amount = 5 False True 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 4 Z = 5 MSE = 1 n n ∑ i=1 ( − )2 = 5 5 yi ̂yi 1 2 3 4 5
42. 42. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 4 fare amount = 5 False True 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 4 Z = 5 MSE = 1 n n ∑ i=1 ( − )2 = 1yi ̂yi 1 2 3 4 5
43. 43. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 4 fare amount = 5 False True 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 4 Z = 5 MSE = 1 n n ∑ i=1 ( − )2 = 1yi ̂yi 1 2 3 4 5 so, if X = 2.5, Y = 4 and Z = 5, MSE is 1
44. 44. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 4 fare amount = 5 False True 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 4 Z = 5 MSE = 1 n n ∑ i=1 ( − )2 = 1yi ̂yi 1 2 3 4 5 Can we ﬁnd better Y and Z? so, if X = 2.5, Y = 4 and Z = 5, MSE is 1
45. 45. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 3 fare amount = 5 False True 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 3 Z = 5 1 2 3 4 5 MSE = 1 n n ∑ i=1 ( − )2 = (y1 − ̂y1)2 + (y2 − ̂y2)2 + (y3 − ̂y3)2 + (y4 − ̂y4)2 + (y5 − ̂y5)2 5 yi ̂yi
46. 46. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 3 fare amount = 5 False True 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 3 Z = 5 1 2 3 4 5 MSE = 1 n n ∑ i=1 ( − )2 = (2 − 3)2 + (4 − 3)2 + (5 − 5)2 + (4 − 5)2 + (5 − 5)2 5 yi ̂yi
47. 47. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 3 fare amount = 5 False True 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 3 Z = 5 1 2 3 4 5 MSE = 1 n n ∑ i=1 ( − )2 = 1 + 1 + 0 + 1 + 0 5 yi ̂yi
48. 48. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 3 fare amount = 5 False True 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 3 Z = 5 1 2 3 4 5 MSE = 1 n n ∑ i=1 ( − )2 = 3 5 = 0.6yi ̂yi
49. 49. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 3 fare amount = 5 False True 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 3 Z = 5 1 2 3 4 5 MSE = 1 n n ∑ i=1 ( − )2 = 3 5 = 0.6yi ̂yi so, if X = 2.5, Y = 3 and Z = 5, MSE is 0.6
50. 50. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 3 fare amount = 4.5 False True 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 3 Z = 4.66 1 2 3 4 5 MSE = 1 n n ∑ i=1 ( − )2 = (2 − 3)2 + (4 − 3)2 + (5 − 4.66)2 + (4 − 4.66)2 + (5 − 4.66)2 5 yi ̂yi
51. 51. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 3 fare amount = 4.5 False True 2.5 What are most reasonable values for Y and Z (that minimise total MSE)? Y = 3 1 2 3 4 5 MSE = 1 n n ∑ i=1 ( − )2 = 1 + 1 + 0.12 + 0.43 + 0.12 5 yi ̂yi Z = 4.66
52. 52. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 3 fare amount = 4.5 False True 2.5 Y = 3 1 2 3 4 5 MSE = 1 n n ∑ i=1 ( − )2 = 2.67 5 = 0.53yi ̂yi so, if Y = 3 and Z = 4.5, MSE is smallest Are we happy? Z = 4.66
53. 53. Decision Tree Algorithm Is distance > 2.5 fare amount = 3 fare amount = 4.5 False True Hold on, how did we choose this split on the ﬁrst place? y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 2.5 1 2 3 4 5
54. 54. Decision Tree Algorithm Is distance > 2.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 3 fare amount = 4.5 False True 2.5 1 2 3 4 5 Hold on, how did we choose this split on the ﬁrst place? Maybe there are better options?
55. 55. Decision Tree Algorithm Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True 1 2 3 4 5 What are the possible split options in this case?
56. 56. Decision Tree Algorithm Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True 1 2 3 4 5 What are the possible split options in this case? 0.5 1.5 2.5 3.5 4.5 5.5
57. 57. Decision Tree Algorithm Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True 1 2 3 4 5 Are these meaningful? 0.5 5.5
58. 58. Decision Tree Algorithm Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True 1 2 3 4 5 How to compare remaining? 1.5 2.5 3.5 4.5
59. 59. Decision Tree Algorithm Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True 1 2 3 4 5 How to compare remaining? For each one we can compute MSE ?? ? ?MSE 1.5 2.5 3.5 4.5
60. 60. Decision Tree Algorithm Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True 1 2 3 4 5 How to compare remaining? For each one we can compute MSE 0.53? ? ? 1.5 2.5 3.5 4.5 MSE
61. 61. Decision Tree Algorithm Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True 1 2 3 4 5 ? Y = 2 Z = 4.5 1.5 MSE
62. 62. Decision Tree Algorithm Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True 1 2 3 4 5 (0 + 0.25 + 0.25 + 0.25 + 0.25)/5 = 0.2 Y = 2 Z = 4.5 1.5 MSE
63. 63. Decision Tree Algorithm Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True 1 2 3 4 5 How to compare remaining? For each one we can compute MSE 0.2 ? ? 1.5 2.5 3.5 4.5 MSE 0.53
64. 64. Decision Tree Algorithm Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True 1 2 3 4 5 ? 3.5 MSE Y = 3.66 Z = 4.5
65. 65. Decision Tree Algorithm Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True 1 2 3 4 5 1.03 3.5 MSE Y = 3.66 Z = 4.5
66. 66. Decision Tree Algorithm Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True 1 2 3 4 5 How to compare remaining? For each one we can compute MSE 0.2 1.03 ? 1.5 2.5 3.5 4.5 MSE 0.53
67. 67. Decision Tree Algorithm Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True 1 2 3 4 5 ? 4.5 MSE Y = 3.75 Z = 5
68. 68. Decision Tree Algorithm Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True 1 2 3 4 5 0.95 4.5 MSE Y = 3.75 Z = 5
69. 69. Decision Tree Algorithm Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True 1 2 3 4 5 How to compare remaining? For each one we can compute MSE 0.2 1.03 0.95 1.5 2.5 3.5 4.5 MSE 0.53
70. 70. Decision Tree Algorithm Is distance > X y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = Y fare amount = Z False True 1 2 3 4 5 We choose the split that minimises total MSE 0.2 1.03 0.95 1.5 2.5 3.5 4.5 MSE 0.53
71. 71. Decision Tree Algorithm Is distance > 1.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 2 fare amount = 4.5 False True 1 2 3 4 5 Thus, the resulting tree: 0.2 1.5 MSE
72. 72. Decision Tree Algorithm Is distance > 1.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 fare amount = 2 fare amount = 4.5 False True 1 2 3 4 5 Can we make our decision tree more accurate? 0.2 1.5 MSE
73. 73. Decision Tree Algorithm distance > 1.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 False True 1 2 3 4 5 Can we make our decision tree more accurate? 0.2 1.5 MSE Yes, by going deeper! fare amount = 2 distance > X fare amount = Y fare amount = Z False True
74. 74. Decision Tree Algorithm distance > 1.5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 False True 1 2 3 4 5 Can we make our decision tree more accurate? 0.2 1.5 MSE Yes, by going deeper! fare amount = 2 distance > X fare amount = Y fare amount = Z False True Let’s return to our Colabs
75. 75. Overﬁtting y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 y X fareamount distance 2 3 4 5 6 1 1 2 3 4 5 Simple, but imperfect Complicated, but ideal VS
76. 76. Train/val split Initial dataset MSE = 1.0 Train dataset Randomly select 60% MSE = 0.0 Simple, but imperfect Complicated, but ideal Validation (val) dataset Randomly select 40% MSE = 2.5 MSE = 0.5
77. 77. POINTS
78. 78. POINTS 1. MACHINE LEARNING MODEL IS NOT MAGIC 2. YOU CAN SAVE AND LOAD ML MODELS 3. EVALUATING MODEL PERFORMANCE IS IMPORTANT 4. YOU MAY NEED TO RETRAIN YOUR MODELS
79. 79. THANK YOU