Explains the problems with ConvNets and Introduces Capsule Neural Networks in simple words.
References and Further reading -
1. https://arxiv.org/abs/1609.08758
2. https://arxiv.org/abs/1710.08864
3. https://arxiv.org/abs/1710.09829v1
4. https://medium.com/mlreview/deep-neural-network-capsules-137be2877d44
Thanks to -
https://www.youtube.com/watch?v=VKoLGnq15RM&t=1099s
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
Problems with CNNs and Introduction to capsule neural networks
1. Problems with CNNs and Introduction to Capsule
Neural Networks
Vipul Vaibhaw
2. Talk Overview
● Why Neural Networks?
● Introduction to Convolutional Neural Networks
● The Problems with Convolutional Neural Networks
● Introduction to Capsule Neural Networks
3. Why Neural Networks?
1. Different approach than conventional algorithmic approach
2. Great for identifying patterns.
3. More generalized solutions.
4. Neural networks and conventional algorithmic computers are not in
competition but complement each other.
4. Convolutional Neural Networks
In machine learning, a convolutional
neural network (CNN, or ConvNet) is a
class of deep, feed-forward artificial
neural networks that has successfully
been applied to analyzing visual
imagery.
11. No Relationship b/w nose and mouth!
Sub-sampling/Pooling of the images loses the relationship between higher
level parts such as Nose and Mouth.
These spatial relationship are very much needed for identity recognition
This is not a face!
12. Equivariance v/s Invariance
● Pooling is invariant to small changes in viewpoints.
● Equivariance - Changes in viewpoints leads to corresponding changes in
Neural Networks.
13. Fooling Deep Neural Nets!
● Deep neural networks(DNN) is not continuous and very sensitive to tiny
perturbation on the input vectors.
● CNN perform poorly when there is Noise in the image.
● Say, changing 1 pixel in an image won’t have any effect on a picture of Cat
to a human being but when this attack was carried on a DNN its
confidence dropped from 98.7% to 73.8%
Link to the research paper - https://arxiv.org/abs/1710.08864
16. What is a capsule?
● A capsule is a group of neurons whose activity vector represents the
instantiation parameters of a specific type of entity such as an object or
object part.
● It nests a new layer inside a layer.
● Instead of making a layer deeper in height, it makes a layer deeper in a
structure.
17. More about Capsules
● Capsules are like cortical columns in human brains.
● Capsules are supposed to produce equivariant features, like a 3D graphic
model: given the model with just a simple transformation we can derive
all its poses
18. Dynamic Routing in Capsule NN
Research paper published by Hinton - https://arxiv.org/abs/1710.09829v1
19. The cost of this new architecture?
● The data flow is more complicated
● No idea how stable it will be for attacking difficult learning problems.
● That makes it harder to calculate gradients, and the model may suffer
more from vanishing gradients.
● Scalability?
20. Conclusion
● ConvNets is proven to solve many real world problems but it has its own
drawbacks.
● Capsule Nets are a promising development to ConvNets.
● It is too early to predict the success of Capsule Nets because it is yet to be
implemented on datasets other than MNIST dataset.
It is just a quick introduction to Neural Networks.
2. For the last ten years Neural networks have attracted a great deal of attention. They offer an alternative approach to computing and to understanding of the human brain. This approach is not something new. The first artificial neuron was produced in 1943 by the neurophysiologist Warren McCulloch and the logician Walter Pits. During the sixties, for reasons that are out of the scope of this article, people turned away from neural networks and concentrated in the symbolic side of Artificial Intelligence. Only in the eighties scientists saw the real potential of neural networks.
After this slide, show a demo of video summarization.
