Small overview of deep learning for non initiated

1. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Shallow Dive into
Deep Learning
Sam Bessalah - @samklr

2. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
“ This talk is filled with buzzwords and terms,
that might make you sound too nerdy or
pedantic at dinner. ”
Use with caution.

3. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

4. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

5. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
In the news, lately ...

6. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

7. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

8. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

9. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

10. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

11. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Skype Translator Demo
https://www.youtube.com/watch?v=eu9kMIeS0wQ

12. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Google DeepMind's Deep Q-learning playing Atari
https://www.youtube.com/watch?v=V1eYniJ0Rnk

13. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Deep Learning?

14. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Multiple Definitions :
- A set of algorithms that try to model high-level abstractions in data by
using architectures composed of multiple non linear transformations.
- A set of machine learning algorithms that automatically learn features
hierarchies.
- Usually built using Neural Networks.
- Representation Learning: Automatically learning good representations
of the data for your classifier, i.e learn good features
- Deep Learning : Learning multiple levels of representation with
complex layers architectures.

15. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Typical Machine Learning Workflow

16. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Typical Machine Learning Workflow

17. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
A Neural Networks Primer

18. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Loose imitation of the brain neurons

19. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Loose imitation of the brain neurons

20. @YourTwitterHandle#Devoxx #YourTag @samklr#Devoxx #deepLearning
@nivdul
Perceptron

21. @YourTwitterHandle#Devoxx #YourTag @samklr#Devoxx #deepLearning
@nivdul
Perceptron
Output Error computed with E, and Network error
minimized with a stochastic gradient descent.

22. @YourTwitterHandle#Devoxx #YourTag @samklr#Devoxx #deepLearning
Stochastic gradient descent and
Backpropagation
- The optimal value for each weight
where at error achieves a global
minimum.
- Backpropagation : Compute error in
the output, then propagate it back
through the network, to update the
weights during the training phase.

23. @YourTwitterHandle#Devoxx #YourTag @samklr#Devoxx #deepLearning

24. @YourTwitterHandle#Devoxx #YourTag @samklr#Devoxx #deepLearning
@nivdul
Multi-layer Perceptron

25. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Problem with Large Networks
- Vanishing Gradient : More layers kills the back propagation. As
information is passed back; gradient value startts to vanish and
become smaller compared to weights.
- Overfitting : Algorithm fits too closely the training data, but will
fail miserably on real examples

26. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

27. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Problem with Large Networks
- Vanishing Gradient : More layers kills the back propagation. As
information is passed back; gradient value startts to vanish and
become smaller compared to weights.
- Overfitting : Algorithm fits too closely the training data, but will
fail miserably on real examples
This led to another AI
winter

28. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

29. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Why does work suddenly ?
- A lot more labeled data
- More compute power CPU and GPU
- Clever new idea on how to train deep
architectures

30. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Convolution Neural Networks

31. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Convolution Neural Networks

32. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Convolution Neural Networks

33. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

34. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

35. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

36. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Recurrent Neural Networks (RNN)
- Learn from arbitrary sequential inputs, by using their
internal states
- Cannot look far back (back propagation limited)
- Long Short Term Memory Networks help solving this
- Good for NLP, hand writing, etc

37. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Recurrent Neural Networks (RNN)

38. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Deep Visual-Semantic Alignments for Generating Image Descriptions CNN + RNN

39. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Other Techniques
- Auto Encoders
- Restricted Boltzmann Machine (RBM)
- Deep Belief
- Hierarchical temporal memory (HTM)
- LSTM
- More and more ...

40. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

41. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Frameworks and Libraries

42. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

43. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

44. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

45. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
H20.ai Deep Learning

46. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
DeepLearning4j

47. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

48. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Getting Started
- Coursera course on Neural Networks from G. Hinton
- Deeplearning4j.org
- Deeplearning.net

49. @YourTwitterHandle#Devoxx #YourTag @samklr#Devoxx #deepLearning
Deep Dream

50. @YourTwitterHandle#Devoxx #YourTag @samklr#Devoxx #deepLearning
CONCLUSION