2. Agenda
What is the computer?
Introduction to machine learning
Applications on machine learning
Introduction to deep learning
Convolutional Neural Networks
How to build a computer to start deep learning
Deep learning Tools
Companies and deep learning
2
3. What is computer?
A computer is a machine that understands zeros and ones only.
The computer can perform complex mathematical and arithmetic operations
very fast.
It performs this using a combination of circuits and logic gates.
3
4. Comment on the following
What is the result?
50 + 30 ?
4
5. Comment on the following
What is the result?
58945 + 78954?
5
9. Comment on the following
Can you write a program to compute the
power ?
9
10. Comment on the following
Can you write a program to recognize
faces or recognize brands of cars?
10
11. Conclusion
Computers are very powerful and accurate in calculations, they can perform
any arithmetic operation faster than any human being.
Humans are faster and more accurate than computers in recognition tasks.
Do you know any one who could compute 787452665 * 75767487 in just
seconds?
Are the machines capable recognize and to reach human level accuracy in
recognition ?
11
12. What is Machine Learning?
“A computer program is said to learn from experience E with respect to some
task T and some performance measure P, if its performance on T, as
measured by P, improves with experience E.” -- Tom Mitchell, Carnegie
Mellon University
12
13. What is Machine Learning?
Machine learning is a discipline of AI
It is a series of techniques that are used to make the computer smarter .
ML solves problems that cannot be solved by numerical means alone.
Machine learning is empowering a lot of technologies used today.
Among the different types of ML tasks, a crucial distinction is drawn between
supervised and unsupervised learning:
Supervised machine learning: The program is “trained” on a pre-defined set of
“training examples”, which then facilitate its ability to reach an accurate
conclusion when given new data.
Unsupervised machine learning: The program is given a bunch of data and must
find patterns and relationships therein.
13
14. Supervised Machine Learning
In the majority of supervised learning applications, the ultimate goal is to
develop a finely tuned predictor function ℎ(𝑥) (sometimes called the
“hypothesis”)
Given input data 𝑥 about a certain domain (say, square footage of a house), it
will accurately predict some interesting value h 𝑥
In practice, x almost always represents multiple data points. So, for example,
a housing price predictor might take not only square-footage (x1) but also
number of bedrooms (x2), number of bathrooms (x3), number of floors (x4),
year built (x5), zip code (x6), and so forth. Determining which inputs to use is
an important part of ML design. However, for the sake of explanation, it is
easiest to assume a single input value is used.
14
15. So let’s say our simple predictor has this form:
Where 𝜃0 and 𝜃1 are constants. Our goal is to find the perfect values 𝜃0 and
𝜃1 to make our predictor work as well as possible.
To make our program learn 𝜃0 and 𝜃1 we should update them according to
error analysis which is called mean square error.
The weights are updated using gradient decent algorithm 15
18. Unsupervised Learning
Unsupervised learning typically is tasked with finding relationships within
data.
There are no training examples used in this process.
Instead, the system is given a set data and tasked with finding patterns and
correlations therein.
Examples: astronomical data analysis, social network analysis and cocktail
party problem and news grouping
18
19. Applications on machine learning
Spam filtering: Classifying emails as spam or non-spam
Weather forecast: Machine learning is applied in weather forecasting
software to improve the quality of the forecast.
Anti-virus: Machine learning is used in Anti-virus software's to improve
detection of malicious software on computer devices.
Personal Assistants: Siri & Cortana
Classifying a tumor as benign tumor or malignant tumor.
19
20. Deep Learning
Deep Learning is a subfield of machine learning concerned with algorithms
inspired by the structure and function of the brain called artificial neural
networks.
Andrew Ng: “very large neural networks we can now have and … huge
amounts of data that we have access to”
Andrew Ng: “for most flavors of the old generations of learning algorithms …
performance will plateau. … deep learning … is the first class of algorithms …
that is scalable. … performance just keeps getting better as you feed them
more data”
20
22. Deep Learning
Deep learning is set of techniques that in some cases could reach human
accuracy in recognition !
The most common used and the most famous architecture is convolutional
neural network.
There are some other techniques like:
LSTM (Long Short Term Memory)
Residual Neural Networks
Autoencoders
Generative Adversarial Networks.
We will focus on convolution neural networks.
22
23. A toy ConvNet: X’s and O’s
X or OCNN
A two-dimensional
array of pixels
28. -1 -1 -1 -1 -1 -1 -1 -1 -1
-1 X -1 -1 -1 -1 X X -1
-1 X X -1 -1 X X -1 -1
-1 -1 X 1 -1 1 -1 -1 -1
-1 -1 -1 -1 1 -1 -1 -1 -1
-1 -1 -1 1 -1 1 X -1 -1
-1 -1 X X -1 -1 X X -1
-1 X X -1 -1 -1 -1 X -1
-1 -1 -1 -1 -1 -1 -1 -1 -1
What computers see
88. How to build a computer to start deep
learning ?
Deep learning techniques’ are resources hungry.
They require huge computing power to give you good performance.
Imagine we have 10 convolution layers of:
256 Conv Filters * Image(1000, 1000)
88
89. How to build a computer to start deep
learning ?
This is pretty huge !.
If you ran this on a CPU it would take 1 day to process 60 000 training images
on I7 processor!
What if the network is deeper than 20 layers and more complex than 256 conv
filters / layer?
The network in this case may take days or even weeks to finish!!
89
90. So, what is the solution ?
In each layer in a CNN we are convolving the image with different filters.
What if we could convolve the image with different filters parallel in the
same time?
The solution is to use GPU!
90
91. Comment on the following
91
How many cores inside a CPU ?
How many cores inside a GPU?
92. What are GPUs brands available out
there?
There are two famous companies:
Unfortunately, AMD GPUs can’t be used for deep learning.
Nvidia GPUs only can be used
That’s because Nvidia provides tools and support for deep learning geeks.
There is a library called NVIDIA Cuda Toolkit
92
94. Nvidia GPUs
Almost all Nvidia’s GPUs will do the job.
You must check if it is CUDA enabled or not.
You can check online. Just search for CUDA supported GPUs.
Some of CUDA supported GPUs: TITAN X, Geforce GTX 1080, Geforce GTX 1070,
Geforce GTX 1060, Geforce GTX 1050, Geforce GTX 980, Geforce GTX 970,
Geforce GTX 960, Geforce GTX 9xx M Series, Geforce GTX 6xx Series, Geforce GTX
6xx M Series.
You can always use your CPU if you don’t have Nvidia GPU or if you don’t have
GPU at all!.
94
95. Deep Learning Tools
After building your computer and configuring it for deep learning, you need
the right tools and APIs to start coding
There are plenty of languages that can be used:
Matlab
Python
C++
C#
Java
And so many…
95
96. Deep Learning Tools
Deep learning frameworks:
NVIDIA CUDA programming APIs.
Tensorflow (Google’s library) for python.
Theano for python.
Torch (Facebook’s library) for Lua.
Caffe for python.
CNTK (Microsoft’s library) for python.
Keras to simplify coding for tensorflow and theano.
96
97. Deep Learning and Companies
Many Companies use deep learning on a daily basis.
Facebook: facebook auto tagger, videos and photos auto caption, post
analyzer,
DeepMind (acquired by google): AlphaGo and many
Amazon
97
99. References
Coursera’s Machine Learning course by Andrew Ng
CS231n: Convolutional Neural Networks for Visual Recognition
Setup a Deep Learning Environment on Windows (Theano & Keras with
GPU Enabled)
ConvNets Visualization
How do Convolutional Neural Networks work?
Rana el Kaliouby, Co-founder, CEO at Affectiva
Hussein Mehanna
99