In this talk we will introduce artificial neural networks and their similarities to how the brain works. We will provide some history and theoretical foundations to introduce feed-forward multi-layer neural networks, describing their predictive and learning ability, and some of their applications in the real world. This is part one of an introductory pair of talks on deep learning concepts and theory.
Valentino Zocca has a Ph.D. in Mathematics from the University of Maryland with a thesis in theoretical geometry, though he has since worked on technical applications and first-on-the block VR geo-navigation data tools and data analysis. Currently he lives in Italy and the United States where he works on emerging deep learning protocols and neural network architectures.
Neural Networks and Deep Learning (Part 1 of 2): An introduction - Valentino Zocca, Real Data Machines
1. An introduction to neural nets
by Valentino Zocca
vzocca@realdatamachines.com
vzocca@gmail.com
+39 333 789 2692
+1 202 640 1381
2. Deep Learning in AI
Digitalisation
Features
Classification/Prediction
AnalysisInteraction
The World
Sensor
Machine
Learning
3. 1642
• Pascaline: first mechanical adder, invented by
French mathematician Blaise Pascal, using a
system of gears and wheels could add and
subtract numbers.
4. 1694
• Machine by Gottfried Wilhelm Von Leibniz
who also developed calculus and invented the
binary system. His machine could also
multiplicate and divide.
5. 1801
• Joseph Marie Charles invents the Jacquard
loom to weave different patterns using cards
punched with holes. Precursor for modern
computers and data storage.
6. 1890
• Herman Hollerith, founder of the Tabulating
Machine Company (later merged into IBM),
creates a mechanical tabulator using punched
cards to store data to calculate statistics. He is
regarded as the father of modern machine
data processing.
7. 1957
• Frank Rosenblatt invents the perceptron
algorithm, the first neural networks
implementation. It was later proved by Marvin
Minsky and Seymour Papert in 1969 that it
could not learn the XOR function.
8. 1974
• Paul Werbos’s Ph.D. thesis describes the
process of training neural nets through back-
propagation.
9. Supervised Learning
1. Input Data
2. Process the information
3. Check the output
4. Learn new rule
5. New rule is applied to better performance
10. Neural Networks
The theory of neural networks arises
from the attempt to mimic our
biological brain in order to create
machines that can learn or perform
pattern recognition in order to make
predictions.
12. Neural Networks
In neural networks, a space of
”weights” is defined alongside the
input space. The weights and the
input together define the activity
rules that in turn will define the
output according to specified
activation functions.
13. Neural Networks
The weights can change with time as
the neural network learns and their
change may be specified by some
learning rule which will generally be
depending on the activities of the
neurons.
14. Perspective
• Biggest artificial neural network to-date: Over
11 billion parameters. (1.1 * 10ˆ10).
• Number of neurons in brain 10ˆ11, each with
about 10ˆ4 connections for a total of 10ˆ15
parameters.
15. A model for a neuron
Images from upcoming book "Python Deep Learning"
21. The Universal Approximation Theorem
Neural networks with a single
hidden layer can be used to
approximate any continuous
function to any desired precision.