Here, Jérémie Kalfon www.jkobject.com presents a presentation review of the paper: zero shot learning throught cross modal transfer

1. Zero-Shot Learning
Cognitive Neural Networks
Jérémie Kalfon
Kent University

2. Table of contents
1. Introduction
2. Concept
3. How it Works
4. Results
5. Discussion
6. Conclusion
1”Zero-Shot Learning Through Cross-Modal Transfer” Richard Socher, Milind Ganjoo, Christopher D. Manning, Andrew Y. Ng
1

3. Introduction

4. Requisites
Neural Networks
• can reduce dimensionality
• can represent information
• Auto Encoders are unsupervised
• How close is it to the brain’s inner working ?
• What can we do with this idea ?
2

5. Problematic
• How to improve neural nets capacity to learn ?
• the brain can learn new things
• zero shot work with unknown data
• How do we reproduce this capacity ?
3

6. Concept

7. Intuition
Figure 1: High dimensional data
4

8. Intuition
Figure 2: High dimensional data with labels
5

9. How it Works

10. presentation
image and words... 2vec the cookbook :
• label region of a feature space
• map the vector image to this space
• but match them together
• How do we do that ?
6

11. The Model
three networks :
Figure 3: auto-encoder
7

12. The Model
three networks :
Figure 4: Neural network
8

13. The Model
three networks :
Figure 4: Neural network
min
Θ
J =
∑
Vw
∑
VI
||(Ww − θ2 × tanh(θ1XI)||2
(1)
8

14. The Model
what happens in the neural net ?
9

15. The Model
what happens in the neural net ?
Figure 5: manifold of known classes
9

16. The Model
what happens in the neural net ?
Figure 5: manifold of known classes
does it belong to a known classes ?
9

17. Detection Strategy
The first strategy is :
• pass the image vector in the neural net
• draw a multinomial around each class
• accept in class if over threshold
10

18. Detection Strategy
The first strategy is :
• pass the image vector in the neural net
• draw a multinomial around each class
• accept in class if over threshold
The second is :
• find the k nearest points.
• compute the probability to be an outlier using the LoOP method
• accept if above threshold
10

19. Classification Strategy
Now we know something about whether it is seen or unseen :
• Seen : find the right class using a regular softmax classifier
• Unseen : find the closest class
11

20. Results

21. method choice
Figure 6: comparison of the two methods
12

22. efficiency
(a) unseen class (b) with distractor words
13

23. Discussion

24. Discussion
• it is a very flexible idea
• much more has to be explored.
• topological learning
• Can it be a hint at how the brain is computing ?
14

25. Conclusion

26. Questions?
14

27. Plottings
Figure 8: manifold of known classes
15

28. Plottings
Figure 9: High dimensional data with labels
16