Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Decision trees and random forests

Decision trees and random forests

  • Inicia sesión para ver los comentarios

Decision trees and random forests

  1. 1. Decision Trees and Random Forests Dr. Debdoot Sheet Assistant Professor Department of Electrical Engineering Indian Institute of Technology Kharagpur www.facweb.iitkgp.ernet.in/~debdoot/
  2. 2. NOT ABOUT WALKING IN A FOREST 1 Mar 2015 2Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  3. 3. IS ALL ABOUT 1 Mar 2015 3Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  4. 4. Overview • Historical Perspective • Decision Tree • Random Forest • Application Scenarios • Computational Complexity • Variable Importance • What’s hot about them in ML Research? 1 Mar 2015 4Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  5. 5. Historical Perspective Decision Trees • L. Breiman, J. Friedman, C. J. Stone, and R. A. Olshen, Classification and Regression Trees. Chapman and Hall/CRC (SIAM), 1984. • J. R. Quinlan, C4.5: Programs for Machine Learning. 1993. Random Forests • Y. Amit and D. Geman., “Shape quantization and recognition with randomized trees,” Neural Computation, vol. 9, pp. 1545– 1588, 1997. • T. K. Ho, “The random subspace method for constructing decision forests,” IEEE T-PAMI, vol. 20, no. 8, pp. 832–844, 1998. • L. Breiman, “Random forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001. 1 Mar 2015 5Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  6. 6. DECISION TREE 1 Mar 2015 6Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  7. 7. Problem Statement 1 Mar 2015 7Decision Trees and Random Forests / Debdoot Sheet / MLCN2015 Formica rufa (Red wood ant)
  8. 8. Classification vs. Regression 1 Mar 2015 8Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  9. 9. Decision Tree 1 Mar 2015 9Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  10. 10. Forming a Decision Tree 1 Mar 2015 10Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  11. 11. Step 1: Split Function at Node Axis aligned split Oblique split Polynomial split 1 Mar 2015 11Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  12. 12. Step 2: Assessing Purity of Split 1 Mar 2015 12Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  13. 13. Cost function for Split Purity Entropy of class distribution Information Gain 1 Mar 2015 13Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  14. 14. Step 3: Selecting Optimum Split  1   2   3   k  1f 2f nf Max. info. gain 1 Mar 2015 14Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  15. 15. Step 4: Stopping Criteria 1 Mar 2015 15Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  16. 16. Step 5: Leaf Prediction Model 1 Mar 2015 16Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  17. 17. Deploying a Decision Tree 1 Mar 2015 17Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  18. 18. RANDOM FOREST 1 Mar 2015 18Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  19. 19. Growing Multiple Trees in a Forest Bagging – Bootstrapped Aggregation 1 Mar 2015 19Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  20. 20. Ensemble Prediction Model 1 Mar 2015 20Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  21. 21. What do we gain by using a Forest? 1 Mar 2015 21Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  22. 22. Noise Resilience and Topology Independence 1 Mar 2015 22Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  23. 23. Effect of Tree Depth 1 Mar 2015 23Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  24. 24. Effect of Split Function 1 Mar 2015 24Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  25. 25. Classification Margin 1 Mar 2015 25Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  26. 26. Random Forest vs. AdaBoost 1 Mar 2015 26Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  27. 27. Random Forest vs. SVM 1 Mar 2015 27Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  28. 28. Regression Forest 1 Mar 2015 28Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  29. 29. Manifold Forest 1 Mar 2015 29Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  30. 30. APPLICATION SCENARIOS After the Brainstorming (Break)! 1 Mar 2015 30Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  31. 31. Gaming – Kinect for Xbox 360 Depth map Body part classification J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, and A. Blake, “Real-time human pose recognition in parts from a single depth image,” in Proc. CVPR, 2011. 1 Mar 2015 31Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  32. 32. Vision – Scene Classification Bosch, A., Zisserman, A., & Muoz, X. “Image classification using random forests and ferns”, ICCV 2007. 1 Mar 2015 32Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  33. 33. A. Criminisi, D. Robertson, E. Konukoglu, J. Shotton, S. Pathak, S. White, and K. Siddiqui, ”Regression Forests for Efficient Anatomy Detection and Localization in Computed Tomography Scans”, Medical Image Analysis, 2013 Medical – Digital Anatomy 1 Mar 2015 33Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  34. 34. Medical – Computational Histology 1 Mar 2015 Decision Trees and Random Forests / Debdoot Sheet / MLCN2015 34 D. Sheet, et al., “Iterative self-organizing atherosclerotic tissue labeling in intravascular ultrasound images and comparison with virtual histology”, IEEE TBME, 59(11), 2012 Hunting for necrosis in the shadows of intravascular ultrasound, CMIG, 38(2), 2014 D. Sheet et al., “Joint learning of ultrasonic backscattering statistical physics and signal confidence primal for characterizing atherosclerotic plaques using intravascular ultrasound”, Med. Image Anal,18(1), 2014 D. Sheet, et al, “In situ histology of mice skin through transfer learning of tissue energy interaction in optical coherence tomography”, J. Biomed. Optics, 18(9), 2013. D. Sheet, et al., “Transfer Learning of Tissue Photon Interaction in Optical Coherence Tomography towards In vivo Histology of the Oral Mucosa”, Proc. Int. Symp. Biomed. Imaging (ISBI), 2014. SPK Karri and D. Sheet, et al., “Computational Histology of Retina through Transfer Learning of Tissue Photon Interaction in Optical Coherence Tomography”, Proc. Int. Symp. Biomed. Imaging (ISBI), 2014. SPK Karri and D Sheet et al., “Deep Learnt Random Forests for Segmentation of Retinal Layers in Optical Coherence Tomography Images”, Int. Symp. Biomedical Imaging (ISBI), 2014. K Basak and D Sheet et al., “Learning of Tissue Photon Interaction in Laser Speckle Contrast Imaging for Label-free Retinal Angiography”, Int. Symp. Biomed. Imaging (ISBI), 2014 D Sheet, et al, “Detection of retinal vessels in fundus images through transfer learning of tissue specific photon interaction statistical physics”, Proc. Int. Symp. Biomed. Imaging (ISBI), 2013.
  35. 35. ENGINEERING DESIGN PERSPECTIVE 1 Mar 2015 35Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  36. 36. Understanding Computations 1 Mar 2015 36Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  37. 37. Computational Complexity Training Complexity Testing Complexity 1 Mar 2015 37Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  38. 38. Features and their Role Feature 1 Feature2 1 Mar 2015 38Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  39. 39. Variable Importance Genuer, R., Poggi, J.-M., Tuleau-Malot, C., (2010). Variable selection using random forests. Pat. Recog. Letters. 31(14):2225-2236 1 Mar 2015 39Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  40. 40. WHAT’S HOT IN RESEARCH? 1 Mar 2015 40Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  41. 41. Research Challenges in 2015 • Architecture – Online learning – Incremental learning – Long term memory – Parallel - distributed architectures – Split functions, cost functions, stopping criteria – Domain adaptation • Engineering and Application – Computational complexity • Statistics and Science – Consistency of forests – VC dimension – De-correlated trees 1 Mar 2015 41Decision Trees and Random Forests / Debdoot Sheet / MLCN2015
  42. 42. Take Home Message • Reading – L. Breiman, J. Friedman, C. J. Stone, and R. A. Olshen, Classification and Regression Trees. Chapman and Hall/CRC, 1984. – L. Breiman, “Random forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001. – A. Criminisi and J. Shotton, Decision Forests for Computer Vision and Medical Image Analysis, Springer, 2013. • Toolboxes and Packages – randomForest in R – TreeBagger in Matlab – sklearn.ensemble.RandomFo restClassifier in Python- Scikit-learn • Conferences – Int. Conf. Comp. Vis. (ICCV) – Eur. Conf. Comp. Vis. (ECCV) – Asian Conf. Comp. Vis. (ACCV) – Comp. Vis. Patt. Recog. (CVPR) – Med. Image Comp., Comp. Assist. Interv. (MICCAI) 1 Mar 2015 42Decision Trees and Random Forests / Debdoot Sheet / MLCN2015

×