Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Deep Learning: AI Breakthrough

792 visualizaciones

Publicado el

A short talk on principles of video processing and deep learning

Publicado en: Ciencias
  • Sé el primero en comentar

Deep Learning: AI Breakthrough

  1. 1. Deep Learning: AI Breakthrough Mohsen Fayyaz Sensifai Tehran University – 15 Dey 1395 (4 Jan 2017)
  2. 2. Video Processing and Deep Learning
  3. 3. What is Video? • Batches of Frames • Can we process video as batches of frames? Motion cannot be inferred from single frame
  4. 4. Why do we need video processing? • Self-Driving Cars: Video Semantic Segmentation Feature Space Optimization for Semantic Video Segmentation, Kundu et. al., 2016
  5. 5. Why do we need video processing? • Robots: Action Recognition Simonyan et. al., 2014
  6. 6. Why do we need video processing? • Google, YouTube, Aparat : Video Tagging Densecap, Johnson et. al., 2016 (Image captioning)
  7. 7. Why do we need video processing? • Network Video Broadcasting: Frame Prediction Patraucean et. al., 2016
  8. 8. From Images to Video 3 Image CNN Extracted Features Frames ? Extracted Features Image Video
  9. 9. From Images to Video CNN Extracted Spatio-Temporal Features Frames LSTM Donahe et. al., 2015
  10. 10. From Images to Video CNN Extracted Spatio-Temporal Features Frames LSTM Donahe et. al., 2015 What if we want regional features?
  11. 11. From Images to Video - STFCN CNN Extracted Regional Spatio-Temporal FeaturesFrames Convolutional LSTM Fayyaz et. al., 2016
  12. 12. From Images to Video – C3D 3D CNN Extracted Regional Spatio-Temporal FeaturesFrames Tran et. al., 2015
  13. 13. Now that we have the appropriate tool Let’s see some real world applications
  14. 14. Video Semantic Segmentation - STFCN Fayyaz et. al., 2016
  15. 15. Video Semantic Segmentation – C3D Tran et. al., 2015
  16. 16. Action Recognition & Video Classification Simonyan et. al., 2014
  17. 17. Does video have visual data only?
  18. 18. Action Recognition & Video Classification Wu et al., 2015 Audio + Vision
  19. 19. Let’s briefly take a look at some state-of-the- art Image based Networks
  20. 20. Extremely Deep Networks Residual Networks • Problem: Gradients Vanish in Back-propagation • Solution: Let’s make a shortcut for them! • Y = 𝐻(𝑋, 𝑊𝐻) -> Y = 𝐻 𝑋, 𝑊𝐻 + 𝑋
  21. 21. Extremely Deep Networks Highway Networks • Similar to ResNets • The shortcuts are controlled using a learnable parameter to have a better trade-off between being • Y = 𝐻 𝑋, 𝑊𝐻 . 𝑇 𝑋, 𝑊𝑇 + 𝑋. (1 − 𝑇 𝑋, 𝑊𝑇 )
  22. 22. Extremely Deep Networks DenseNets • If ResNet works with just connecting previous layers, why not connecting all?! • 𝑌 = 𝐹(𝑋 𝑛, 𝑋 𝑛−1, …, 𝑋0) • Improvements in both Forward & • Backward
  23. 23. Now what if we use the idea of propagating data and gradients between shallow and deep layers in video based networks?
  24. 24. Up to here everything was Supervised But there are bunch of data across the Internet with weak labels … Let’s go through Weakly-Supervised methods
  25. 25. Weakly Supervised Learning Weakly Supervised Learning with CNNs • Multiple Labeling • Weakly Localization • Data can be crawled over Internet • Can be adopted to Video Oquab et. al., 2015
  26. 26. How about some Unsupervised methods …
  27. 27. Unsupervised Learning Anticipating Visual Representations From Unlabeled Video • Training on Big Huge amount of unlabeled Video across the net • Training Classifiers on the final output Vondrick et. al., 2016
  28. 28. Practical considerations
  29. 29. What Hardware do I use? • NVIDIA GPU + SSD + HDD • More info on: http://www.DeepLearning.ir
  30. 30. What framework do I use? Caffe Torch Tensorflow Theano Keras Microsoft CNTK Deeplearning4j …
  31. 31. What framework do I use? Tensorflow Torch Theano From Karpathy’s slides
  32. 32. Distributed Training: Will be presented at my next presentation at Sharif University of Technology on 22 Dey 1395 (11 Jan 2017) From Karpathy’s slides
  33. 33. Thank You Fayyaz@Sensifai.com

×