SlideShare una empresa de Scribd logo
1 de 25
Descargar para leer sin conexión
Workshop: Dendritic integration and
computation with active dendrites
February 8, 2018
Subutai Ahmad
sahmad@numenta.com
The Predictive Neuron: How Active Dendrites Enable
Spatiotemporal Computation In The Neocortex
1)			Understand	operating	principles of	the	neocortex
- seek	biologically	derived	theories
- test	empirically	and	via	simulation
2)			Enable	technology	based	on	cortical	principles
- active	open	source	community
- intelligent	machines	will	be	based	on	cortical	principles
Observation:
The neocortex is constantly predicting its inputs.
“the most important and also the most neglected problem of
cerebral physiology” (Lashley, 1951)
How can networks of pyramidal neurons learn predictive
models of the world?
Research question:
1) How can neurons learn predictive models of extrinsic temporal sequences?
3) Experimentally testable predictions
- Impact of NMDA spikes
- Branch specific plasticity
- Sparse correlation structure
- Pyramidal neuron uses active dendrites for prediction
- A single layer network model for complex predictions
- Works on real world applications
- Extension of sequence memory model
- Learns models of objects using motion based context signal
- Can predict sensory features using movement
“Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in the Neocortex”
Hawkins and Ahmad, Frontiers in Neural Circuits, 2016/03/30
“A Theory of How Columns in the Neocortex Learn the Structure of the World”
Hawkins, Ahmad, and Cui, Frontiers in Neural Circuits, 2017/10/25
2) How can neurons learn predictive models of sensorimotor sequences?
5K to 30K excitatory synapses
- 10% proximal
- 90% distal
Distal dendrites are pattern detectors
- 8-15 co-active, co-located synapses
generate dendritic NMDA spikes
- sustained depolarization of soma but
does not typically generate AP
Pyramidal Neuron
(Mel, 1992; Branco & Häusser, 2011; Schiller et al, 2000; Losonczy, 2006; Antic
et al, 2010; Major et al, 2013; Spruston, 2008; Milojkovic et al, 2005, etc.)
Prediction Starts in the Neuron
Proximal synapses: Cause somatic spikes
Define classic receptive field of neuron
Distal synapses: Cause dendritic spikes
Put the cell into a depolarized, or “predictive” state
Depolarized neurons fire sooner, inhibiting nearby neurons.
A neuron can predict its activity in hundreds of unique contexts.
5K to 30K excitatory synapses
- 10% proximal
- 90% distal
Distal dendrites are pattern detectors
- 8-15 co-active, co-located synapses
generate dendritic NMDA spikes
- sustained depolarization of soma but
does not typically generate AP
HTM Neuron Model
Prediction Starts in the Neuron
Pyramidal Neuron
(Poirazi et al., 2003)
(Hawkins & Ahmad, 2016)
A Single Layer Network Model for Sequence Memory
- Neurons in a mini-column learn same FF receptive field.
- Active dendritic segments form connections to nearby cells.
- Depolarized cells fire first, and inhibit other cells within mini-column.
No prediction Predicted input
(Hawkins & Ahmad, 2016)
(Cui et al, 2016)
t=0
t=1
Predicted cells inhibit
neighbors
Next prediction t=2
t=0
t=1
Synaptic changes localized to dendritic segments:
(Stuart and Häusser, 2001; Losonczy et al., 2008)
1. If a cell was correctly predicted, positively reinforce the dendritic
segment that caused the prediction.
2. If a cell was incorrectly predicted, slightly negatively reinforce the
corresponding dendritic segment.
3. If no cell was predicted in a mini-column, reinforce the dendritic
segment that best matched the previous input.
Continuous Branch Specific Learning
X
A B
B
C
C
D
Y
Before learning
X B’’ C’’
D’
Y’’
After learning
A B’ C’
Same columns,
but only one cell active per column.
High Order (Non-Markovian) Sequences
Two sequences: A-B-C-D
X-B-C-Y
Application To Real World Streaming Data Sources
- Accuracy is comparable to state of the art ML techniques (LSTM, ARIMA, etc.)
- Continuous unsupervised learning - adapts to changes far better than other techniques
- Top benchmark score in detecting anomalies and unusual behavior
- Extremely fault tolerant (tolerant to 40% noise and faults)
- Multiple open source implementations (some commercial)
“Continuous online sequence learning with an unsupervised neural network model”
Cui, Ahmad and Hawkins, Neural Computation, 2016
“Unsupervised real-time anomaly detection for streaming data”
Ahmad, Lavin, Purdy and Zuha, Neurocomputing, 2017
2015-04-20
Monday
2015-04-21
Tuesday
2015-04-22
Wednesday
2015-04-23
Thursday
2015-04-24
Friday
2015-04-25
Saturday
2015-04-26
Sunday
0 k
5 k
10 k
15 k
20 k
25 k
30 k
PassengerCountin30minwindow
A
B C
Shift
AR
IM
ALSTM
1000LSTM
3000LSTM
6000
TM
0.0
0.2
0.4
0.6
0.8
1.0
NRMSE
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
MAPE
0.0
0.5
1.0
1.5
2.0
2.5
NegativeLog-likelihood
Shift
AR
IM
ALSTM
1000LSTM
3000LSTM
6000
TM
LSTM
1000LSTM
3000LSTM
6000
TM
D
?
NYC	Taxi	Demand Machine	Temperature	Sensor	Data
1) How can neurons learn predictive models of temporal sequences?
3) Experimentally testable predictions
- Impact of NMDA spikes
- Branch specific plasticity
- Sparse correlation structure
- Pyramidal neuron uses active dendrites for prediction
- A single layer network model for complex predictions
- Works on real world applications
- Extension of sequence memory model
- Learns models of objects using motion based context signal
- Can predict sensory features using movement
“Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in the Neocortex”
Hawkins and Ahmad, Frontiers in Neural Circuits, 2016/03/30
“A Theory of How Columns in the Neocortex Learn the Structure of the World”
Hawkins, Ahmad, and Cui, Frontiers in Neural Circuits, 2017/10/25
2) How can neurons learn predictive models of sensorimotor sequences?
How Could Neurons Learn Predictive Models of Sensorimotor
Sequences?
Sequence memory
Sensorimotor sequences
SensorMotor-related context
Hypothesis:
- We need to provide an “allocentric location”, updated through movement
- Grid cell modules, applied to objects, can compute allocentric location
Two Layer Model of Sensorimotor Inference
Feature @ location
Object Stable over movement of sensor
- Using allocentric location, cellular layers can accurately predict its
input as the sensor moves.
Sensor
Feature
Allocentric
Location
Pooling
Seq Mem
Changes with each movement
Yale-CMU-Berkeley (YCB) Object Benchmark (Calli et al, 2017)
- 80 objects designed for robotics grasping tasks
- Includes high-resolution 3D CAD files
YCB Object Benchmark
We created a virtual hand using the Unity game engine
Curvature based sensor on each fingertip
4096 neurons per layer per column
98.7% recall accuracy (77/78 uniquely classified)
Convergence time depends on object, sequence of
sensations, number of fingers.
Simulation using YCB Object Benchmark
Pairwise confusion between objects after 1 touch
Convergence 1 finger 1 touch
Pairwise confusion between objects after 2 touches
Convergence 1 finger 2 touches
Pairwise confusion between objects after 6 touches
Convergence 1 finger 6 touches
Pairwise confusion between objects after 10 touches
Convergence 1 finger 10 touches
1) How can neurons learn predictive models of temporal sequences?
3) Experimentally testable predictions
- Impact of NMDA spikes
- Branch specific plasticity
- Sparse correlation structure
- Pyramidal neuron uses active dendrites for prediction
- A single layer network model for complex predictions
- Works on real world applications
- Extension of sequence memory model
- Learns models of objects using motion based context signal
- Can predict sensory features using movement
“Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in the Neocortex”
Hawkins and Ahmad, Frontiers in Neural Circuits, 2016/03/30
“A Theory of How Columns in the Neocortex Learn the Structure of the World”
Hawkins, Ahmad, and Cui, Frontiers in Neural Circuits, 2017/10/25
2) How can neurons learn predictive models of sensorimotor sequences?
1)	Impact	of	NMDA	spikes:
Dendritic	NMDA	spikes	cause	cells	to	fire	faster	than	they	would	otherwise.
Fast	local	inhibitory	networks	(e.g.	minicolumns)	inhibit	cells	that	don’t	fire	early.
Sparser	activations	during	a	predictable	sensory	stream.	
For	predictable	natural	stimuli,	dendritic	spikes	will	be	more	frequent	than	APs.	
(Vinje &	Gallant,	2002;	Smith	et	al,	2013;	Wilmes et	al,	2016;	Moore	et	al,	2017)
2)	Branch	specific	plasticity:
Strong	LTP	in	dendritic	branch	when	NMDA	spike	followed	by	back	action	potential	(bAP).	
Weak	LTP	(without	NMDA	spike)	if	synapse	cluster	becomes	active	followed	by	a	bAP.
Weak	LTD	when	an	NMDA	spike	is	not	followed	by	an	action	potential/bAP.
(Holthoff et	al,	2004;	Losonczy et	al,	2008;	Yang	et	al,	2014;	Cichon &	Gang,	2015)
3)	Correlation	structure:
Low	pair-wise	correlations	between	cells	but	significant	high-order	correlations.
High	order	assembly	correlated	with	specific	point	in	a	predictable	sequence.
Unanticipated	inputs	leads	to	a	burst	of	activity,	correlated	within minicolumns.
Activity	during	predicted	inputs	will	be	a	subset	of	activity	during	unpredicted	inputs.
Neighboring	mini-columns	will	be	uncorrelated.
(Ecker	et	al,	2010;	Smith	&	Häusser,	2010;	Schneidman	et	al,	2006;	Miller	et	al,	2014;	Homann et	al,	2017)
Properties And Experimentally Testable Predictions
20
Correlation Structure With Natural Sequences
Cell Assembly Order
3 4 5 6
Numberofunique
cellassemblies
x104
0
1
2
3
4
5
6
7
Trial Number
0 5 10 15 20
ActivationDensity
# 10-3
3
4
5
6
7
8
9
10
11
12
V1
AL
D
ataShuffled
Poisson
Subsetindex
0
0.02
0.04
0.06
0.08
V1
Trial 1 Trial 20
Visual
Stimulus
Screen
A
Time (sec)
5 10 15 20 25 30
Neuron#
0
50
100
150
200
250
300
350
Time (sec)
5 10 15 20 25 30
Neuron#
0
50
100
150
200
250
300
350
Neuron#
80
100
120
140
160
Neuron#
80
100
120
140
160
Cell A
3 4
0
500
1000
1500
2000
2500
3000V1
ALV1
V1
AL
B
C
F
E
D
0.2
0.3
0.06
0.08
serving
lassembly
serving
lassembly
(Stirman	et	al,	2016)
Spencer	L.	Smith YiYi Yu
20 presentations of a 30-
second natural movie
Sparser Activity With Repeated Presentations
Cell Assembly Order
3 4 5 6
Numberofunique
cellassemblies
x104
0
1
2
3
4
5
6
7
Trial Number
0 5 10 15 20
Ac
3
4
5
6
0
D
ataShuffled
Poisson
0
0.02
Trial 1 Trial 20
Stimulus
Screen
Time (sec)
5 10 15 20 25 30
Neuron#
0
50
100
150
200
250
300
350
Time (sec)
5 10 15 20 25 30
Neuron#
0
50
100
150
200
250
300
350
Time (sec)
5 10 15 20 25 30
Neuron#
0
20
40
60
80
100
120
140
160
Time (sec)
5 10 15 20 25 30
Neuron#
0
20
40
60
80
100
120
140
160
Cell Asse
3 4
0
500
1000
1500
2000
2500
3000 ALV1
ALV1
V1
AL
-1 -0.5 0 0.5 1
0
0.1
0.2
0.3
-1 -0.5 0 0
0
0.02
0.04
0.06
0.08
Time jitter (sec)
Prob.ofobserving
repeatedcellassembly
Time jitter (sec
Prob.ofobserving
repeatedcellassembly
Cell Assembly Order
3 4 5 6
Numberofunique
cellassemblies
x104
0
1
2
3
4
5
6
7
Trial Number
0 5 10 15 20
ActivationDensity
# 10-3
3
4
5
6
7
8
9
10
11
12
V1
AL
Subsetindex
0.0
0.0
0.0
0.0
Trial 1 Trial 20
Visual
Stimulus
Screen
Time (sec)
5 10 15 20 25 30
Neuron#
0
50
100
150
200
250
300
350
Time (sec)
5 10 15 20 25 30
0
500
1000
1500
2000
2500
3000V1
V1
Similar to (Vinje & Gallant, 2002)
Emergence of High Order Cell Assemblies
Cell Assembly Order
3 4 5 6
Numberofunique
cellassemblies
x104
0
1
2
3
4
5
6
7
Trial Number
0 5 10 15 20
3
4 0
D
ataShuffled
Poisson
0
30
30
Cell Assembly Order
3 4 5 6
0
500
1000
1500
2000
2500
3000
Data
Shuffled
Poisson
D
ataShuffled
Poisson
ALV1
ALV1
3-order cell
assembly
single cell
-1 -0.5 0 0.5 1
0
0.1
0.2
0.3
-1 -0.5 0 0.5 1
0
0.02
0.04
0.06
0.08
Time jitter (sec)
Prob.ofobserving
repeatedcellassembly
Time jitter (sec)
Prob.ofobserving
repeatedcellassembly
Cell assemblies are significantly more likely to
occur in sequences than predicted by a
Poisson model (p<0.001).
Cell Assembly Order
3 4 5 6
Numberofunique
cellassemblies
x104
0
1
2
3
4
5
6
7
Trial Number
0 5 10 15 20
3
0
D
ataShuffled
Poisson
0
Cell Assembly Order
3 4 5 6
0
500
1000
1500
2000
2500
3000
Data
Shuffled
Poisson
D
ataShuffled
Poisson
ALV1
ALV1
3-order cell
assembly
single cell
-1 -0.5 0 0.5 1
0
0.1
0.2
0.3
-1 -0.5 0 0.5 1
0
0.02
0.04
0.06
0.08
Time jitter (sec)
Prob.ofobserving
repeatedcellassembly
Time jitter (sec)
Prob.ofobserving
repeatedcellassembly
Sparse code predicts specific point in a
sequence (single cells don’t).
Similar to (Miller et al, 2014)
- A model of sequence learning in cortex
- Relies on “predictive neuron” with active dendrites and fast inhibitory networks
- Can learn complex temporal sequences
- Applied to real world streaming applications
- A model of sensorimotor sequence learning in cortex
- Identical network of pyramidal cells can perform a very different computation
- Difference: motion related context sent to active dendrites
- Detailed list of experimentally testable properties
24
Summary
Open Issues / Discussion
Are active dendrites necessary? (Yes!)
- Is a two layer network of uniform point neurons sufficient? (No!)
How to integrate calcium spikes, BAC firing, and apical dendrites?
Continuous time model of HTM, including inhibitory networks
Collaborations
We are always interested in hosting visiting scholars and interns.
Co-authors: Jeff Hawkins, Scott Purdy, Marcus Lewis (Numenta), Yuwei Cui
Contact info: sahmad@numenta.com
@SubutaiAhmad

Más contenido relacionado

La actualidad más candente

Jeff Hawkins Human Brain Project Summit Keynote: "Location, Location, Locatio...
Jeff Hawkins Human Brain Project Summit Keynote: "Location, Location, Locatio...Jeff Hawkins Human Brain Project Summit Keynote: "Location, Location, Locatio...
Jeff Hawkins Human Brain Project Summit Keynote: "Location, Location, Locatio...Numenta
 
Sparsity In The Neocortex, And Its Implications For Machine Learning
Sparsity In The Neocortex,  And Its Implications For Machine LearningSparsity In The Neocortex,  And Its Implications For Machine Learning
Sparsity In The Neocortex, And Its Implications For Machine LearningNumenta
 
CVPR 2020 Workshop: Sparsity in the neocortex, and its implications for conti...
CVPR 2020 Workshop: Sparsity in the neocortex, and its implications for conti...CVPR 2020 Workshop: Sparsity in the neocortex, and its implications for conti...
CVPR 2020 Workshop: Sparsity in the neocortex, and its implications for conti...Christy Maver
 
Location, Location, Location - A Framework for Intelligence and Cortical Comp...
Location, Location, Location - A Framework for Intelligence and Cortical Comp...Location, Location, Location - A Framework for Intelligence and Cortical Comp...
Location, Location, Location - A Framework for Intelligence and Cortical Comp...Numenta
 
Have We Missed Half of What the Neocortex Does? A New Predictive Framework ...
 Have We Missed Half of What the Neocortex Does?  A New Predictive Framework ... Have We Missed Half of What the Neocortex Does?  A New Predictive Framework ...
Have We Missed Half of What the Neocortex Does? A New Predictive Framework ...Numenta
 
BAAI Conference 2021: The Thousand Brains Theory - A Roadmap for Creating Mac...
BAAI Conference 2021: The Thousand Brains Theory - A Roadmap for Creating Mac...BAAI Conference 2021: The Thousand Brains Theory - A Roadmap for Creating Mac...
BAAI Conference 2021: The Thousand Brains Theory - A Roadmap for Creating Mac...Numenta
 
Locations in the Neocortex: A Theory of Sensorimotor Prediction Using Cortica...
Locations in the Neocortex: A Theory of Sensorimotor Prediction Using Cortica...Locations in the Neocortex: A Theory of Sensorimotor Prediction Using Cortica...
Locations in the Neocortex: A Theory of Sensorimotor Prediction Using Cortica...Numenta
 
Why Neurons have thousands of synapses? A model of sequence memory in the brain
Why Neurons have thousands of synapses? A model of sequence memory in the brainWhy Neurons have thousands of synapses? A model of sequence memory in the brain
Why Neurons have thousands of synapses? A model of sequence memory in the brainNumenta
 
Principles of Hierarchical Temporal Memory - Foundations of Machine Intelligence
Principles of Hierarchical Temporal Memory - Foundations of Machine IntelligencePrinciples of Hierarchical Temporal Memory - Foundations of Machine Intelligence
Principles of Hierarchical Temporal Memory - Foundations of Machine IntelligenceNumenta
 
Fundamentals of Neural Networks
Fundamentals of Neural NetworksFundamentals of Neural Networks
Fundamentals of Neural NetworksGagan Deep
 
What the Brain says about Machine Intelligence
What the Brain says about Machine Intelligence What the Brain says about Machine Intelligence
What the Brain says about Machine Intelligence Numenta
 
Neural Networks
Neural Networks Neural Networks
Neural Networks Eric Su
 
Hierarchical Temporal Memory: Computing Like the Brain - Matt Taylor, Numenta
Hierarchical Temporal Memory: Computing Like the Brain - Matt Taylor, NumentaHierarchical Temporal Memory: Computing Like the Brain - Matt Taylor, Numenta
Hierarchical Temporal Memory: Computing Like the Brain - Matt Taylor, NumentaWithTheBest
 
neural networks
 neural networks neural networks
neural networksjoshiblog
 
Recognizing Locations on Objects by Marcus Lewis
Recognizing Locations on Objects by Marcus LewisRecognizing Locations on Objects by Marcus Lewis
Recognizing Locations on Objects by Marcus LewisNumenta
 

La actualidad más candente (20)

Jeff Hawkins Human Brain Project Summit Keynote: "Location, Location, Locatio...
Jeff Hawkins Human Brain Project Summit Keynote: "Location, Location, Locatio...Jeff Hawkins Human Brain Project Summit Keynote: "Location, Location, Locatio...
Jeff Hawkins Human Brain Project Summit Keynote: "Location, Location, Locatio...
 
Sparsity In The Neocortex, And Its Implications For Machine Learning
Sparsity In The Neocortex,  And Its Implications For Machine LearningSparsity In The Neocortex,  And Its Implications For Machine Learning
Sparsity In The Neocortex, And Its Implications For Machine Learning
 
CVPR 2020 Workshop: Sparsity in the neocortex, and its implications for conti...
CVPR 2020 Workshop: Sparsity in the neocortex, and its implications for conti...CVPR 2020 Workshop: Sparsity in the neocortex, and its implications for conti...
CVPR 2020 Workshop: Sparsity in the neocortex, and its implications for conti...
 
Location, Location, Location - A Framework for Intelligence and Cortical Comp...
Location, Location, Location - A Framework for Intelligence and Cortical Comp...Location, Location, Location - A Framework for Intelligence and Cortical Comp...
Location, Location, Location - A Framework for Intelligence and Cortical Comp...
 
Have We Missed Half of What the Neocortex Does? A New Predictive Framework ...
 Have We Missed Half of What the Neocortex Does?  A New Predictive Framework ... Have We Missed Half of What the Neocortex Does?  A New Predictive Framework ...
Have We Missed Half of What the Neocortex Does? A New Predictive Framework ...
 
BAAI Conference 2021: The Thousand Brains Theory - A Roadmap for Creating Mac...
BAAI Conference 2021: The Thousand Brains Theory - A Roadmap for Creating Mac...BAAI Conference 2021: The Thousand Brains Theory - A Roadmap for Creating Mac...
BAAI Conference 2021: The Thousand Brains Theory - A Roadmap for Creating Mac...
 
Locations in the Neocortex: A Theory of Sensorimotor Prediction Using Cortica...
Locations in the Neocortex: A Theory of Sensorimotor Prediction Using Cortica...Locations in the Neocortex: A Theory of Sensorimotor Prediction Using Cortica...
Locations in the Neocortex: A Theory of Sensorimotor Prediction Using Cortica...
 
Why Neurons have thousands of synapses? A model of sequence memory in the brain
Why Neurons have thousands of synapses? A model of sequence memory in the brainWhy Neurons have thousands of synapses? A model of sequence memory in the brain
Why Neurons have thousands of synapses? A model of sequence memory in the brain
 
Principles of Hierarchical Temporal Memory - Foundations of Machine Intelligence
Principles of Hierarchical Temporal Memory - Foundations of Machine IntelligencePrinciples of Hierarchical Temporal Memory - Foundations of Machine Intelligence
Principles of Hierarchical Temporal Memory - Foundations of Machine Intelligence
 
Fundamentals of Neural Networks
Fundamentals of Neural NetworksFundamentals of Neural Networks
Fundamentals of Neural Networks
 
What the Brain says about Machine Intelligence
What the Brain says about Machine Intelligence What the Brain says about Machine Intelligence
What the Brain says about Machine Intelligence
 
Nencki321 day2
Nencki321 day2Nencki321 day2
Nencki321 day2
 
Neural Networks
Neural Networks Neural Networks
Neural Networks
 
Neural network
Neural networkNeural network
Neural network
 
Hierarchical Temporal Memory: Computing Like the Brain - Matt Taylor, Numenta
Hierarchical Temporal Memory: Computing Like the Brain - Matt Taylor, NumentaHierarchical Temporal Memory: Computing Like the Brain - Matt Taylor, Numenta
Hierarchical Temporal Memory: Computing Like the Brain - Matt Taylor, Numenta
 
Lec 1-2-3-intr.
Lec 1-2-3-intr.Lec 1-2-3-intr.
Lec 1-2-3-intr.
 
neural networks
 neural networks neural networks
neural networks
 
intelligent system
intelligent systemintelligent system
intelligent system
 
Recognizing Locations on Objects by Marcus Lewis
Recognizing Locations on Objects by Marcus LewisRecognizing Locations on Objects by Marcus Lewis
Recognizing Locations on Objects by Marcus Lewis
 
Neural networks
Neural networksNeural networks
Neural networks
 

Similar a The Predictive Neuron: How Active Dendrites Enable Spatiotemporal Computation in the Neocortex by Subutai Ahmad (02/08/2018)

CARLsim 3: Concepts, Tools, and Applications
CARLsim 3: Concepts, Tools, and ApplicationsCARLsim 3: Concepts, Tools, and Applications
CARLsim 3: Concepts, Tools, and ApplicationsMichael Beyeler
 
SF Big Analytics20170706: What the brain tells us about the future of streami...
SF Big Analytics20170706: What the brain tells us about the future of streami...SF Big Analytics20170706: What the brain tells us about the future of streami...
SF Big Analytics20170706: What the brain tells us about the future of streami...Chester Chen
 
CVPR 2020 Workshop: Sparsity in the neocortex, and its implications for conti...
CVPR 2020 Workshop: Sparsity in the neocortex, and its implications for conti...CVPR 2020 Workshop: Sparsity in the neocortex, and its implications for conti...
CVPR 2020 Workshop: Sparsity in the neocortex, and its implications for conti...Numenta
 
Artificial Neural Network Abstract
Artificial Neural Network AbstractArtificial Neural Network Abstract
Artificial Neural Network AbstractAnjali Agrawal
 
Neural Networks with Anticipation: Problems and Prospects
Neural Networks with Anticipation: Problems and ProspectsNeural Networks with Anticipation: Problems and Prospects
Neural Networks with Anticipation: Problems and ProspectsSSA KPI
 
Neural Computing
Neural ComputingNeural Computing
Neural ComputingESCOM
 
Artificial Neural Network and its Applications
Artificial Neural Network and its ApplicationsArtificial Neural Network and its Applications
Artificial Neural Network and its Applicationsshritosh kumar
 
Neural networks in business forecasting
Neural networks in business forecastingNeural networks in business forecasting
Neural networks in business forecastingAmir Shokri
 
Artificial Intelligence (AI) Essay
Artificial Intelligence (AI) EssayArtificial Intelligence (AI) Essay
Artificial Intelligence (AI) EssayLindsay Alston
 
Artificial Neural Networks Lect1: Introduction & neural computation
Artificial Neural Networks Lect1: Introduction & neural computationArtificial Neural Networks Lect1: Introduction & neural computation
Artificial Neural Networks Lect1: Introduction & neural computationMohammed Bennamoun
 
Neuromorphic computing for neural networks
Neuromorphic computing for neural networksNeuromorphic computing for neural networks
Neuromorphic computing for neural networksClaudio Gallicchio
 
MUSA HURANTA MSHELIA.pptx
MUSA HURANTA MSHELIA.pptxMUSA HURANTA MSHELIA.pptx
MUSA HURANTA MSHELIA.pptxMusaMshelia4
 
Neural net NWU 4.3 Graphics Course
Neural net NWU 4.3 Graphics CourseNeural net NWU 4.3 Graphics Course
Neural net NWU 4.3 Graphics CourseMohaiminur Rahman
 
Application Of Multi Layer Artificial Neural Network In The Diagnosis System ...
Application Of Multi Layer Artificial Neural Network In The Diagnosis System ...Application Of Multi Layer Artificial Neural Network In The Diagnosis System ...
Application Of Multi Layer Artificial Neural Network In The Diagnosis System ...Richard Hogue
 
Fuzzy Logic Final Report
Fuzzy Logic Final ReportFuzzy Logic Final Report
Fuzzy Logic Final ReportShikhar Agarwal
 
INTRODUCTION TO NEURAL NETWORKS
INTRODUCTION TO NEURAL NETWORKSINTRODUCTION TO NEURAL NETWORKS
INTRODUCTION TO NEURAL NETWORKSPrashant Srivastav
 

Similar a The Predictive Neuron: How Active Dendrites Enable Spatiotemporal Computation in the Neocortex by Subutai Ahmad (02/08/2018) (20)

CARLsim 3: Concepts, Tools, and Applications
CARLsim 3: Concepts, Tools, and ApplicationsCARLsim 3: Concepts, Tools, and Applications
CARLsim 3: Concepts, Tools, and Applications
 
SF Big Analytics20170706: What the brain tells us about the future of streami...
SF Big Analytics20170706: What the brain tells us about the future of streami...SF Big Analytics20170706: What the brain tells us about the future of streami...
SF Big Analytics20170706: What the brain tells us about the future of streami...
 
CVPR 2020 Workshop: Sparsity in the neocortex, and its implications for conti...
CVPR 2020 Workshop: Sparsity in the neocortex, and its implications for conti...CVPR 2020 Workshop: Sparsity in the neocortex, and its implications for conti...
CVPR 2020 Workshop: Sparsity in the neocortex, and its implications for conti...
 
Artificial Neural Network Abstract
Artificial Neural Network AbstractArtificial Neural Network Abstract
Artificial Neural Network Abstract
 
Neural Networks with Anticipation: Problems and Prospects
Neural Networks with Anticipation: Problems and ProspectsNeural Networks with Anticipation: Problems and Prospects
Neural Networks with Anticipation: Problems and Prospects
 
SoftComputing5
SoftComputing5SoftComputing5
SoftComputing5
 
Neural Computing
Neural ComputingNeural Computing
Neural Computing
 
Artificial Neural Network and its Applications
Artificial Neural Network and its ApplicationsArtificial Neural Network and its Applications
Artificial Neural Network and its Applications
 
Neural networks in business forecasting
Neural networks in business forecastingNeural networks in business forecasting
Neural networks in business forecasting
 
Artificial Intelligence (AI) Essay
Artificial Intelligence (AI) EssayArtificial Intelligence (AI) Essay
Artificial Intelligence (AI) Essay
 
Artificial Neural Networks Lect1: Introduction & neural computation
Artificial Neural Networks Lect1: Introduction & neural computationArtificial Neural Networks Lect1: Introduction & neural computation
Artificial Neural Networks Lect1: Introduction & neural computation
 
Neuromorphic computing for neural networks
Neuromorphic computing for neural networksNeuromorphic computing for neural networks
Neuromorphic computing for neural networks
 
ANN.pptx
ANN.pptxANN.pptx
ANN.pptx
 
08 neural networks(1).unlocked
08 neural networks(1).unlocked08 neural networks(1).unlocked
08 neural networks(1).unlocked
 
MUSA HURANTA MSHELIA.pptx
MUSA HURANTA MSHELIA.pptxMUSA HURANTA MSHELIA.pptx
MUSA HURANTA MSHELIA.pptx
 
Neural net NWU 4.3 Graphics Course
Neural net NWU 4.3 Graphics CourseNeural net NWU 4.3 Graphics Course
Neural net NWU 4.3 Graphics Course
 
Application Of Multi Layer Artificial Neural Network In The Diagnosis System ...
Application Of Multi Layer Artificial Neural Network In The Diagnosis System ...Application Of Multi Layer Artificial Neural Network In The Diagnosis System ...
Application Of Multi Layer Artificial Neural Network In The Diagnosis System ...
 
Neural networks
Neural networksNeural networks
Neural networks
 
Fuzzy Logic Final Report
Fuzzy Logic Final ReportFuzzy Logic Final Report
Fuzzy Logic Final Report
 
INTRODUCTION TO NEURAL NETWORKS
INTRODUCTION TO NEURAL NETWORKSINTRODUCTION TO NEURAL NETWORKS
INTRODUCTION TO NEURAL NETWORKS
 

Más de Numenta

Deep learning at the edge: 100x Inference improvement on edge devices
Deep learning at the edge: 100x Inference improvement on edge devicesDeep learning at the edge: 100x Inference improvement on edge devices
Deep learning at the edge: 100x Inference improvement on edge devicesNumenta
 
Brains@Bay Meetup: A Primer on Neuromodulatory Systems - Srikanth Ramaswamy
Brains@Bay Meetup: A Primer on Neuromodulatory Systems - Srikanth RamaswamyBrains@Bay Meetup: A Primer on Neuromodulatory Systems - Srikanth Ramaswamy
Brains@Bay Meetup: A Primer on Neuromodulatory Systems - Srikanth RamaswamyNumenta
 
Brains@Bay Meetup: How to Evolve Your Own Lab Rat - Thomas Miconi
Brains@Bay Meetup: How to Evolve Your Own Lab Rat - Thomas MiconiBrains@Bay Meetup: How to Evolve Your Own Lab Rat - Thomas Miconi
Brains@Bay Meetup: How to Evolve Your Own Lab Rat - Thomas MiconiNumenta
 
Brains@Bay Meetup: The Increasing Role of Sensorimotor Experience in Artifici...
Brains@Bay Meetup: The Increasing Role of Sensorimotor Experience in Artifici...Brains@Bay Meetup: The Increasing Role of Sensorimotor Experience in Artifici...
Brains@Bay Meetup: The Increasing Role of Sensorimotor Experience in Artifici...Numenta
 
Brains@Bay Meetup: Open-ended Skill Acquisition in Humans and Machines: An Ev...
Brains@Bay Meetup: Open-ended Skill Acquisition in Humans and Machines: An Ev...Brains@Bay Meetup: Open-ended Skill Acquisition in Humans and Machines: An Ev...
Brains@Bay Meetup: Open-ended Skill Acquisition in Humans and Machines: An Ev...Numenta
 
Brains@Bay Meetup: The Effect of Sensorimotor Learning on the Learned Represe...
Brains@Bay Meetup: The Effect of Sensorimotor Learning on the Learned Represe...Brains@Bay Meetup: The Effect of Sensorimotor Learning on the Learned Represe...
Brains@Bay Meetup: The Effect of Sensorimotor Learning on the Learned Represe...Numenta
 
SBMT 2021: Can Neuroscience Insights Transform AI? - Lawrence Spracklen
SBMT 2021: Can Neuroscience Insights Transform AI? - Lawrence SpracklenSBMT 2021: Can Neuroscience Insights Transform AI? - Lawrence Spracklen
SBMT 2021: Can Neuroscience Insights Transform AI? - Lawrence SpracklenNumenta
 
FPGA Conference 2021: Breaking the TOPS ceiling with sparse neural networks -...
FPGA Conference 2021: Breaking the TOPS ceiling with sparse neural networks -...FPGA Conference 2021: Breaking the TOPS ceiling with sparse neural networks -...
FPGA Conference 2021: Breaking the TOPS ceiling with sparse neural networks -...Numenta
 
OpenAI’s GPT 3 Language Model - guest Steve Omohundro
OpenAI’s GPT 3 Language Model - guest Steve OmohundroOpenAI’s GPT 3 Language Model - guest Steve Omohundro
OpenAI’s GPT 3 Language Model - guest Steve OmohundroNumenta
 
The Thousand Brains Theory: A Framework for Understanding the Neocortex and B...
The Thousand Brains Theory: A Framework for Understanding the Neocortex and B...The Thousand Brains Theory: A Framework for Understanding the Neocortex and B...
The Thousand Brains Theory: A Framework for Understanding the Neocortex and B...Numenta
 
The Biological Path Toward Strong AI by Matt Taylor (05/17/18)
The Biological Path Toward Strong AI by Matt Taylor (05/17/18)The Biological Path Toward Strong AI by Matt Taylor (05/17/18)
The Biological Path Toward Strong AI by Matt Taylor (05/17/18)Numenta
 
The Biological Path Towards Strong AI Strange Loop 2017, St. Louis
The Biological Path Towards Strong AI Strange Loop 2017, St. LouisThe Biological Path Towards Strong AI Strange Loop 2017, St. Louis
The Biological Path Towards Strong AI Strange Loop 2017, St. LouisNumenta
 
HTM Spatial Pooler
HTM Spatial PoolerHTM Spatial Pooler
HTM Spatial PoolerNumenta
 
Biological path toward strong AI
Biological path toward strong AIBiological path toward strong AI
Biological path toward strong AINumenta
 
Predictive Analytics with Numenta Machine Intelligence
Predictive Analytics with Numenta Machine IntelligencePredictive Analytics with Numenta Machine Intelligence
Predictive Analytics with Numenta Machine IntelligenceNumenta
 

Más de Numenta (15)

Deep learning at the edge: 100x Inference improvement on edge devices
Deep learning at the edge: 100x Inference improvement on edge devicesDeep learning at the edge: 100x Inference improvement on edge devices
Deep learning at the edge: 100x Inference improvement on edge devices
 
Brains@Bay Meetup: A Primer on Neuromodulatory Systems - Srikanth Ramaswamy
Brains@Bay Meetup: A Primer on Neuromodulatory Systems - Srikanth RamaswamyBrains@Bay Meetup: A Primer on Neuromodulatory Systems - Srikanth Ramaswamy
Brains@Bay Meetup: A Primer on Neuromodulatory Systems - Srikanth Ramaswamy
 
Brains@Bay Meetup: How to Evolve Your Own Lab Rat - Thomas Miconi
Brains@Bay Meetup: How to Evolve Your Own Lab Rat - Thomas MiconiBrains@Bay Meetup: How to Evolve Your Own Lab Rat - Thomas Miconi
Brains@Bay Meetup: How to Evolve Your Own Lab Rat - Thomas Miconi
 
Brains@Bay Meetup: The Increasing Role of Sensorimotor Experience in Artifici...
Brains@Bay Meetup: The Increasing Role of Sensorimotor Experience in Artifici...Brains@Bay Meetup: The Increasing Role of Sensorimotor Experience in Artifici...
Brains@Bay Meetup: The Increasing Role of Sensorimotor Experience in Artifici...
 
Brains@Bay Meetup: Open-ended Skill Acquisition in Humans and Machines: An Ev...
Brains@Bay Meetup: Open-ended Skill Acquisition in Humans and Machines: An Ev...Brains@Bay Meetup: Open-ended Skill Acquisition in Humans and Machines: An Ev...
Brains@Bay Meetup: Open-ended Skill Acquisition in Humans and Machines: An Ev...
 
Brains@Bay Meetup: The Effect of Sensorimotor Learning on the Learned Represe...
Brains@Bay Meetup: The Effect of Sensorimotor Learning on the Learned Represe...Brains@Bay Meetup: The Effect of Sensorimotor Learning on the Learned Represe...
Brains@Bay Meetup: The Effect of Sensorimotor Learning on the Learned Represe...
 
SBMT 2021: Can Neuroscience Insights Transform AI? - Lawrence Spracklen
SBMT 2021: Can Neuroscience Insights Transform AI? - Lawrence SpracklenSBMT 2021: Can Neuroscience Insights Transform AI? - Lawrence Spracklen
SBMT 2021: Can Neuroscience Insights Transform AI? - Lawrence Spracklen
 
FPGA Conference 2021: Breaking the TOPS ceiling with sparse neural networks -...
FPGA Conference 2021: Breaking the TOPS ceiling with sparse neural networks -...FPGA Conference 2021: Breaking the TOPS ceiling with sparse neural networks -...
FPGA Conference 2021: Breaking the TOPS ceiling with sparse neural networks -...
 
OpenAI’s GPT 3 Language Model - guest Steve Omohundro
OpenAI’s GPT 3 Language Model - guest Steve OmohundroOpenAI’s GPT 3 Language Model - guest Steve Omohundro
OpenAI’s GPT 3 Language Model - guest Steve Omohundro
 
The Thousand Brains Theory: A Framework for Understanding the Neocortex and B...
The Thousand Brains Theory: A Framework for Understanding the Neocortex and B...The Thousand Brains Theory: A Framework for Understanding the Neocortex and B...
The Thousand Brains Theory: A Framework for Understanding the Neocortex and B...
 
The Biological Path Toward Strong AI by Matt Taylor (05/17/18)
The Biological Path Toward Strong AI by Matt Taylor (05/17/18)The Biological Path Toward Strong AI by Matt Taylor (05/17/18)
The Biological Path Toward Strong AI by Matt Taylor (05/17/18)
 
The Biological Path Towards Strong AI Strange Loop 2017, St. Louis
The Biological Path Towards Strong AI Strange Loop 2017, St. LouisThe Biological Path Towards Strong AI Strange Loop 2017, St. Louis
The Biological Path Towards Strong AI Strange Loop 2017, St. Louis
 
HTM Spatial Pooler
HTM Spatial PoolerHTM Spatial Pooler
HTM Spatial Pooler
 
Biological path toward strong AI
Biological path toward strong AIBiological path toward strong AI
Biological path toward strong AI
 
Predictive Analytics with Numenta Machine Intelligence
Predictive Analytics with Numenta Machine IntelligencePredictive Analytics with Numenta Machine Intelligence
Predictive Analytics with Numenta Machine Intelligence
 

Último

biosynthesis of the cell wall and antibiotics
biosynthesis of the cell wall and antibioticsbiosynthesis of the cell wall and antibiotics
biosynthesis of the cell wall and antibioticsSafaFallah
 
001 Case Study - Submission Point_c1051231_attempt_2023-11-23-14-08-42_ABS CW...
001 Case Study - Submission Point_c1051231_attempt_2023-11-23-14-08-42_ABS CW...001 Case Study - Submission Point_c1051231_attempt_2023-11-23-14-08-42_ABS CW...
001 Case Study - Submission Point_c1051231_attempt_2023-11-23-14-08-42_ABS CW...marwaahmad357
 
Identification of Superclusters and Their Properties in the Sloan Digital Sky...
Identification of Superclusters and Their Properties in the Sloan Digital Sky...Identification of Superclusters and Their Properties in the Sloan Digital Sky...
Identification of Superclusters and Their Properties in the Sloan Digital Sky...Sérgio Sacani
 
Exploration Method’s in Archaeological Studies & Research
Exploration Method’s in Archaeological Studies & ResearchExploration Method’s in Archaeological Studies & Research
Exploration Method’s in Archaeological Studies & ResearchPrachya Adhyayan
 
PSP3 employability assessment form .docx
PSP3 employability assessment form .docxPSP3 employability assessment form .docx
PSP3 employability assessment form .docxmarwaahmad357
 
Application of Foraminiferal Ecology- Rahul.pptx
Application of Foraminiferal Ecology- Rahul.pptxApplication of Foraminiferal Ecology- Rahul.pptx
Application of Foraminiferal Ecology- Rahul.pptxRahulVishwakarma71547
 
Krishi Vigyan Kendras - कृषि विज्ञान केंद्र
Krishi Vigyan Kendras - कृषि विज्ञान केंद्रKrishi Vigyan Kendras - कृषि विज्ञान केंद्र
Krishi Vigyan Kendras - कृषि विज्ञान केंद्रKrashi Coaching
 
3.2 Pests of Sorghum_Identification, Symptoms and nature of damage, Binomics,...
3.2 Pests of Sorghum_Identification, Symptoms and nature of damage, Binomics,...3.2 Pests of Sorghum_Identification, Symptoms and nature of damage, Binomics,...
3.2 Pests of Sorghum_Identification, Symptoms and nature of damage, Binomics,...PirithiRaju
 
SUKDANAN DIAGNOSTIC TEST IN PHYSICAL SCIENCE ANSWER KEYY.pdf
SUKDANAN DIAGNOSTIC TEST IN PHYSICAL SCIENCE ANSWER KEYY.pdfSUKDANAN DIAGNOSTIC TEST IN PHYSICAL SCIENCE ANSWER KEYY.pdf
SUKDANAN DIAGNOSTIC TEST IN PHYSICAL SCIENCE ANSWER KEYY.pdfsantiagojoderickdoma
 
Applied Biochemistry feedback_M Ahwad 2023.docx
Applied Biochemistry feedback_M Ahwad 2023.docxApplied Biochemistry feedback_M Ahwad 2023.docx
Applied Biochemistry feedback_M Ahwad 2023.docxmarwaahmad357
 
IB Biology New syllabus B3.2 Transport.pptx
IB Biology New syllabus B3.2 Transport.pptxIB Biology New syllabus B3.2 Transport.pptx
IB Biology New syllabus B3.2 Transport.pptxUalikhanKalkhojayev1
 
KeyBio pipeline for bioinformatics and data science
KeyBio pipeline for bioinformatics and data scienceKeyBio pipeline for bioinformatics and data science
KeyBio pipeline for bioinformatics and data scienceLayne Sadler
 
Role of Herbs in Cosmetics in Cosmetic Science.
Role of Herbs in Cosmetics in Cosmetic Science.Role of Herbs in Cosmetics in Cosmetic Science.
Role of Herbs in Cosmetics in Cosmetic Science.ShwetaHattimare
 
TORSION IN GASTROPODS- Anatomical event (Zoology)
TORSION IN GASTROPODS- Anatomical event (Zoology)TORSION IN GASTROPODS- Anatomical event (Zoology)
TORSION IN GASTROPODS- Anatomical event (Zoology)chatterjeesoumili50
 
SCIENCE 6 QUARTER 3 REVIEWER(FRICTION, GRAVITY, ENERGY AND SPEED).pptx
SCIENCE 6 QUARTER 3 REVIEWER(FRICTION, GRAVITY, ENERGY AND SPEED).pptxSCIENCE 6 QUARTER 3 REVIEWER(FRICTION, GRAVITY, ENERGY AND SPEED).pptx
SCIENCE 6 QUARTER 3 REVIEWER(FRICTION, GRAVITY, ENERGY AND SPEED).pptxROVELYNEDELUNA3
 
Legacy Analysis of Dark Matter Annihilation from the Milky Way Dwarf Spheroid...
Legacy Analysis of Dark Matter Annihilation from the Milky Way Dwarf Spheroid...Legacy Analysis of Dark Matter Annihilation from the Milky Way Dwarf Spheroid...
Legacy Analysis of Dark Matter Annihilation from the Milky Way Dwarf Spheroid...Sérgio Sacani
 
Human brain.. It's parts and function.
Human brain.. It's parts and function. Human brain.. It's parts and function.
Human brain.. It's parts and function. MUKTA MANJARI SAHOO
 
Lehninger_Chapter 17_Fatty acid Oxid.ppt
Lehninger_Chapter 17_Fatty acid Oxid.pptLehninger_Chapter 17_Fatty acid Oxid.ppt
Lehninger_Chapter 17_Fatty acid Oxid.pptSachin Teotia
 
M.Pharm - Question Bank - Drug Delivery Systems
M.Pharm - Question Bank - Drug Delivery SystemsM.Pharm - Question Bank - Drug Delivery Systems
M.Pharm - Question Bank - Drug Delivery SystemsSumathi Arumugam
 

Último (20)

biosynthesis of the cell wall and antibiotics
biosynthesis of the cell wall and antibioticsbiosynthesis of the cell wall and antibiotics
biosynthesis of the cell wall and antibiotics
 
001 Case Study - Submission Point_c1051231_attempt_2023-11-23-14-08-42_ABS CW...
001 Case Study - Submission Point_c1051231_attempt_2023-11-23-14-08-42_ABS CW...001 Case Study - Submission Point_c1051231_attempt_2023-11-23-14-08-42_ABS CW...
001 Case Study - Submission Point_c1051231_attempt_2023-11-23-14-08-42_ABS CW...
 
Identification of Superclusters and Their Properties in the Sloan Digital Sky...
Identification of Superclusters and Their Properties in the Sloan Digital Sky...Identification of Superclusters and Their Properties in the Sloan Digital Sky...
Identification of Superclusters and Their Properties in the Sloan Digital Sky...
 
Exploration Method’s in Archaeological Studies & Research
Exploration Method’s in Archaeological Studies & ResearchExploration Method’s in Archaeological Studies & Research
Exploration Method’s in Archaeological Studies & Research
 
PSP3 employability assessment form .docx
PSP3 employability assessment form .docxPSP3 employability assessment form .docx
PSP3 employability assessment form .docx
 
Application of Foraminiferal Ecology- Rahul.pptx
Application of Foraminiferal Ecology- Rahul.pptxApplication of Foraminiferal Ecology- Rahul.pptx
Application of Foraminiferal Ecology- Rahul.pptx
 
Krishi Vigyan Kendras - कृषि विज्ञान केंद्र
Krishi Vigyan Kendras - कृषि विज्ञान केंद्रKrishi Vigyan Kendras - कृषि विज्ञान केंद्र
Krishi Vigyan Kendras - कृषि विज्ञान केंद्र
 
3.2 Pests of Sorghum_Identification, Symptoms and nature of damage, Binomics,...
3.2 Pests of Sorghum_Identification, Symptoms and nature of damage, Binomics,...3.2 Pests of Sorghum_Identification, Symptoms and nature of damage, Binomics,...
3.2 Pests of Sorghum_Identification, Symptoms and nature of damage, Binomics,...
 
SUKDANAN DIAGNOSTIC TEST IN PHYSICAL SCIENCE ANSWER KEYY.pdf
SUKDANAN DIAGNOSTIC TEST IN PHYSICAL SCIENCE ANSWER KEYY.pdfSUKDANAN DIAGNOSTIC TEST IN PHYSICAL SCIENCE ANSWER KEYY.pdf
SUKDANAN DIAGNOSTIC TEST IN PHYSICAL SCIENCE ANSWER KEYY.pdf
 
Applied Biochemistry feedback_M Ahwad 2023.docx
Applied Biochemistry feedback_M Ahwad 2023.docxApplied Biochemistry feedback_M Ahwad 2023.docx
Applied Biochemistry feedback_M Ahwad 2023.docx
 
IB Biology New syllabus B3.2 Transport.pptx
IB Biology New syllabus B3.2 Transport.pptxIB Biology New syllabus B3.2 Transport.pptx
IB Biology New syllabus B3.2 Transport.pptx
 
KeyBio pipeline for bioinformatics and data science
KeyBio pipeline for bioinformatics and data scienceKeyBio pipeline for bioinformatics and data science
KeyBio pipeline for bioinformatics and data science
 
Role of Herbs in Cosmetics in Cosmetic Science.
Role of Herbs in Cosmetics in Cosmetic Science.Role of Herbs in Cosmetics in Cosmetic Science.
Role of Herbs in Cosmetics in Cosmetic Science.
 
TORSION IN GASTROPODS- Anatomical event (Zoology)
TORSION IN GASTROPODS- Anatomical event (Zoology)TORSION IN GASTROPODS- Anatomical event (Zoology)
TORSION IN GASTROPODS- Anatomical event (Zoology)
 
SCIENCE 6 QUARTER 3 REVIEWER(FRICTION, GRAVITY, ENERGY AND SPEED).pptx
SCIENCE 6 QUARTER 3 REVIEWER(FRICTION, GRAVITY, ENERGY AND SPEED).pptxSCIENCE 6 QUARTER 3 REVIEWER(FRICTION, GRAVITY, ENERGY AND SPEED).pptx
SCIENCE 6 QUARTER 3 REVIEWER(FRICTION, GRAVITY, ENERGY AND SPEED).pptx
 
Legacy Analysis of Dark Matter Annihilation from the Milky Way Dwarf Spheroid...
Legacy Analysis of Dark Matter Annihilation from the Milky Way Dwarf Spheroid...Legacy Analysis of Dark Matter Annihilation from the Milky Way Dwarf Spheroid...
Legacy Analysis of Dark Matter Annihilation from the Milky Way Dwarf Spheroid...
 
Human brain.. It's parts and function.
Human brain.. It's parts and function. Human brain.. It's parts and function.
Human brain.. It's parts and function.
 
Lehninger_Chapter 17_Fatty acid Oxid.ppt
Lehninger_Chapter 17_Fatty acid Oxid.pptLehninger_Chapter 17_Fatty acid Oxid.ppt
Lehninger_Chapter 17_Fatty acid Oxid.ppt
 
M.Pharm - Question Bank - Drug Delivery Systems
M.Pharm - Question Bank - Drug Delivery SystemsM.Pharm - Question Bank - Drug Delivery Systems
M.Pharm - Question Bank - Drug Delivery Systems
 
Cheminformatics tools supporting dissemination of data associated with US EPA...
Cheminformatics tools supporting dissemination of data associated with US EPA...Cheminformatics tools supporting dissemination of data associated with US EPA...
Cheminformatics tools supporting dissemination of data associated with US EPA...
 

The Predictive Neuron: How Active Dendrites Enable Spatiotemporal Computation in the Neocortex by Subutai Ahmad (02/08/2018)

  • 1. Workshop: Dendritic integration and computation with active dendrites February 8, 2018 Subutai Ahmad sahmad@numenta.com The Predictive Neuron: How Active Dendrites Enable Spatiotemporal Computation In The Neocortex
  • 2. 1) Understand operating principles of the neocortex - seek biologically derived theories - test empirically and via simulation 2) Enable technology based on cortical principles - active open source community - intelligent machines will be based on cortical principles
  • 3. Observation: The neocortex is constantly predicting its inputs. “the most important and also the most neglected problem of cerebral physiology” (Lashley, 1951) How can networks of pyramidal neurons learn predictive models of the world? Research question:
  • 4. 1) How can neurons learn predictive models of extrinsic temporal sequences? 3) Experimentally testable predictions - Impact of NMDA spikes - Branch specific plasticity - Sparse correlation structure - Pyramidal neuron uses active dendrites for prediction - A single layer network model for complex predictions - Works on real world applications - Extension of sequence memory model - Learns models of objects using motion based context signal - Can predict sensory features using movement “Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in the Neocortex” Hawkins and Ahmad, Frontiers in Neural Circuits, 2016/03/30 “A Theory of How Columns in the Neocortex Learn the Structure of the World” Hawkins, Ahmad, and Cui, Frontiers in Neural Circuits, 2017/10/25 2) How can neurons learn predictive models of sensorimotor sequences?
  • 5. 5K to 30K excitatory synapses - 10% proximal - 90% distal Distal dendrites are pattern detectors - 8-15 co-active, co-located synapses generate dendritic NMDA spikes - sustained depolarization of soma but does not typically generate AP Pyramidal Neuron (Mel, 1992; Branco & Häusser, 2011; Schiller et al, 2000; Losonczy, 2006; Antic et al, 2010; Major et al, 2013; Spruston, 2008; Milojkovic et al, 2005, etc.) Prediction Starts in the Neuron
  • 6. Proximal synapses: Cause somatic spikes Define classic receptive field of neuron Distal synapses: Cause dendritic spikes Put the cell into a depolarized, or “predictive” state Depolarized neurons fire sooner, inhibiting nearby neurons. A neuron can predict its activity in hundreds of unique contexts. 5K to 30K excitatory synapses - 10% proximal - 90% distal Distal dendrites are pattern detectors - 8-15 co-active, co-located synapses generate dendritic NMDA spikes - sustained depolarization of soma but does not typically generate AP HTM Neuron Model Prediction Starts in the Neuron Pyramidal Neuron (Poirazi et al., 2003) (Hawkins & Ahmad, 2016)
  • 7. A Single Layer Network Model for Sequence Memory - Neurons in a mini-column learn same FF receptive field. - Active dendritic segments form connections to nearby cells. - Depolarized cells fire first, and inhibit other cells within mini-column. No prediction Predicted input (Hawkins & Ahmad, 2016) (Cui et al, 2016) t=0 t=1 Predicted cells inhibit neighbors Next prediction t=2 t=0 t=1
  • 8. Synaptic changes localized to dendritic segments: (Stuart and Häusser, 2001; Losonczy et al., 2008) 1. If a cell was correctly predicted, positively reinforce the dendritic segment that caused the prediction. 2. If a cell was incorrectly predicted, slightly negatively reinforce the corresponding dendritic segment. 3. If no cell was predicted in a mini-column, reinforce the dendritic segment that best matched the previous input. Continuous Branch Specific Learning
  • 9. X A B B C C D Y Before learning X B’’ C’’ D’ Y’’ After learning A B’ C’ Same columns, but only one cell active per column. High Order (Non-Markovian) Sequences Two sequences: A-B-C-D X-B-C-Y
  • 10. Application To Real World Streaming Data Sources - Accuracy is comparable to state of the art ML techniques (LSTM, ARIMA, etc.) - Continuous unsupervised learning - adapts to changes far better than other techniques - Top benchmark score in detecting anomalies and unusual behavior - Extremely fault tolerant (tolerant to 40% noise and faults) - Multiple open source implementations (some commercial) “Continuous online sequence learning with an unsupervised neural network model” Cui, Ahmad and Hawkins, Neural Computation, 2016 “Unsupervised real-time anomaly detection for streaming data” Ahmad, Lavin, Purdy and Zuha, Neurocomputing, 2017 2015-04-20 Monday 2015-04-21 Tuesday 2015-04-22 Wednesday 2015-04-23 Thursday 2015-04-24 Friday 2015-04-25 Saturday 2015-04-26 Sunday 0 k 5 k 10 k 15 k 20 k 25 k 30 k PassengerCountin30minwindow A B C Shift AR IM ALSTM 1000LSTM 3000LSTM 6000 TM 0.0 0.2 0.4 0.6 0.8 1.0 NRMSE 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 MAPE 0.0 0.5 1.0 1.5 2.0 2.5 NegativeLog-likelihood Shift AR IM ALSTM 1000LSTM 3000LSTM 6000 TM LSTM 1000LSTM 3000LSTM 6000 TM D ? NYC Taxi Demand Machine Temperature Sensor Data
  • 11. 1) How can neurons learn predictive models of temporal sequences? 3) Experimentally testable predictions - Impact of NMDA spikes - Branch specific plasticity - Sparse correlation structure - Pyramidal neuron uses active dendrites for prediction - A single layer network model for complex predictions - Works on real world applications - Extension of sequence memory model - Learns models of objects using motion based context signal - Can predict sensory features using movement “Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in the Neocortex” Hawkins and Ahmad, Frontiers in Neural Circuits, 2016/03/30 “A Theory of How Columns in the Neocortex Learn the Structure of the World” Hawkins, Ahmad, and Cui, Frontiers in Neural Circuits, 2017/10/25 2) How can neurons learn predictive models of sensorimotor sequences?
  • 12. How Could Neurons Learn Predictive Models of Sensorimotor Sequences? Sequence memory Sensorimotor sequences SensorMotor-related context Hypothesis: - We need to provide an “allocentric location”, updated through movement - Grid cell modules, applied to objects, can compute allocentric location
  • 13. Two Layer Model of Sensorimotor Inference Feature @ location Object Stable over movement of sensor - Using allocentric location, cellular layers can accurately predict its input as the sensor moves. Sensor Feature Allocentric Location Pooling Seq Mem Changes with each movement
  • 14. Yale-CMU-Berkeley (YCB) Object Benchmark (Calli et al, 2017) - 80 objects designed for robotics grasping tasks - Includes high-resolution 3D CAD files YCB Object Benchmark We created a virtual hand using the Unity game engine Curvature based sensor on each fingertip 4096 neurons per layer per column 98.7% recall accuracy (77/78 uniquely classified) Convergence time depends on object, sequence of sensations, number of fingers. Simulation using YCB Object Benchmark
  • 15. Pairwise confusion between objects after 1 touch Convergence 1 finger 1 touch
  • 16. Pairwise confusion between objects after 2 touches Convergence 1 finger 2 touches
  • 17. Pairwise confusion between objects after 6 touches Convergence 1 finger 6 touches
  • 18. Pairwise confusion between objects after 10 touches Convergence 1 finger 10 touches
  • 19. 1) How can neurons learn predictive models of temporal sequences? 3) Experimentally testable predictions - Impact of NMDA spikes - Branch specific plasticity - Sparse correlation structure - Pyramidal neuron uses active dendrites for prediction - A single layer network model for complex predictions - Works on real world applications - Extension of sequence memory model - Learns models of objects using motion based context signal - Can predict sensory features using movement “Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in the Neocortex” Hawkins and Ahmad, Frontiers in Neural Circuits, 2016/03/30 “A Theory of How Columns in the Neocortex Learn the Structure of the World” Hawkins, Ahmad, and Cui, Frontiers in Neural Circuits, 2017/10/25 2) How can neurons learn predictive models of sensorimotor sequences?
  • 20. 1) Impact of NMDA spikes: Dendritic NMDA spikes cause cells to fire faster than they would otherwise. Fast local inhibitory networks (e.g. minicolumns) inhibit cells that don’t fire early. Sparser activations during a predictable sensory stream. For predictable natural stimuli, dendritic spikes will be more frequent than APs. (Vinje & Gallant, 2002; Smith et al, 2013; Wilmes et al, 2016; Moore et al, 2017) 2) Branch specific plasticity: Strong LTP in dendritic branch when NMDA spike followed by back action potential (bAP). Weak LTP (without NMDA spike) if synapse cluster becomes active followed by a bAP. Weak LTD when an NMDA spike is not followed by an action potential/bAP. (Holthoff et al, 2004; Losonczy et al, 2008; Yang et al, 2014; Cichon & Gang, 2015) 3) Correlation structure: Low pair-wise correlations between cells but significant high-order correlations. High order assembly correlated with specific point in a predictable sequence. Unanticipated inputs leads to a burst of activity, correlated within minicolumns. Activity during predicted inputs will be a subset of activity during unpredicted inputs. Neighboring mini-columns will be uncorrelated. (Ecker et al, 2010; Smith & Häusser, 2010; Schneidman et al, 2006; Miller et al, 2014; Homann et al, 2017) Properties And Experimentally Testable Predictions 20
  • 21. Correlation Structure With Natural Sequences Cell Assembly Order 3 4 5 6 Numberofunique cellassemblies x104 0 1 2 3 4 5 6 7 Trial Number 0 5 10 15 20 ActivationDensity # 10-3 3 4 5 6 7 8 9 10 11 12 V1 AL D ataShuffled Poisson Subsetindex 0 0.02 0.04 0.06 0.08 V1 Trial 1 Trial 20 Visual Stimulus Screen A Time (sec) 5 10 15 20 25 30 Neuron# 0 50 100 150 200 250 300 350 Time (sec) 5 10 15 20 25 30 Neuron# 0 50 100 150 200 250 300 350 Neuron# 80 100 120 140 160 Neuron# 80 100 120 140 160 Cell A 3 4 0 500 1000 1500 2000 2500 3000V1 ALV1 V1 AL B C F E D 0.2 0.3 0.06 0.08 serving lassembly serving lassembly (Stirman et al, 2016) Spencer L. Smith YiYi Yu 20 presentations of a 30- second natural movie
  • 22. Sparser Activity With Repeated Presentations Cell Assembly Order 3 4 5 6 Numberofunique cellassemblies x104 0 1 2 3 4 5 6 7 Trial Number 0 5 10 15 20 Ac 3 4 5 6 0 D ataShuffled Poisson 0 0.02 Trial 1 Trial 20 Stimulus Screen Time (sec) 5 10 15 20 25 30 Neuron# 0 50 100 150 200 250 300 350 Time (sec) 5 10 15 20 25 30 Neuron# 0 50 100 150 200 250 300 350 Time (sec) 5 10 15 20 25 30 Neuron# 0 20 40 60 80 100 120 140 160 Time (sec) 5 10 15 20 25 30 Neuron# 0 20 40 60 80 100 120 140 160 Cell Asse 3 4 0 500 1000 1500 2000 2500 3000 ALV1 ALV1 V1 AL -1 -0.5 0 0.5 1 0 0.1 0.2 0.3 -1 -0.5 0 0 0 0.02 0.04 0.06 0.08 Time jitter (sec) Prob.ofobserving repeatedcellassembly Time jitter (sec Prob.ofobserving repeatedcellassembly Cell Assembly Order 3 4 5 6 Numberofunique cellassemblies x104 0 1 2 3 4 5 6 7 Trial Number 0 5 10 15 20 ActivationDensity # 10-3 3 4 5 6 7 8 9 10 11 12 V1 AL Subsetindex 0.0 0.0 0.0 0.0 Trial 1 Trial 20 Visual Stimulus Screen Time (sec) 5 10 15 20 25 30 Neuron# 0 50 100 150 200 250 300 350 Time (sec) 5 10 15 20 25 30 0 500 1000 1500 2000 2500 3000V1 V1 Similar to (Vinje & Gallant, 2002)
  • 23. Emergence of High Order Cell Assemblies Cell Assembly Order 3 4 5 6 Numberofunique cellassemblies x104 0 1 2 3 4 5 6 7 Trial Number 0 5 10 15 20 3 4 0 D ataShuffled Poisson 0 30 30 Cell Assembly Order 3 4 5 6 0 500 1000 1500 2000 2500 3000 Data Shuffled Poisson D ataShuffled Poisson ALV1 ALV1 3-order cell assembly single cell -1 -0.5 0 0.5 1 0 0.1 0.2 0.3 -1 -0.5 0 0.5 1 0 0.02 0.04 0.06 0.08 Time jitter (sec) Prob.ofobserving repeatedcellassembly Time jitter (sec) Prob.ofobserving repeatedcellassembly Cell assemblies are significantly more likely to occur in sequences than predicted by a Poisson model (p<0.001). Cell Assembly Order 3 4 5 6 Numberofunique cellassemblies x104 0 1 2 3 4 5 6 7 Trial Number 0 5 10 15 20 3 0 D ataShuffled Poisson 0 Cell Assembly Order 3 4 5 6 0 500 1000 1500 2000 2500 3000 Data Shuffled Poisson D ataShuffled Poisson ALV1 ALV1 3-order cell assembly single cell -1 -0.5 0 0.5 1 0 0.1 0.2 0.3 -1 -0.5 0 0.5 1 0 0.02 0.04 0.06 0.08 Time jitter (sec) Prob.ofobserving repeatedcellassembly Time jitter (sec) Prob.ofobserving repeatedcellassembly Sparse code predicts specific point in a sequence (single cells don’t). Similar to (Miller et al, 2014)
  • 24. - A model of sequence learning in cortex - Relies on “predictive neuron” with active dendrites and fast inhibitory networks - Can learn complex temporal sequences - Applied to real world streaming applications - A model of sensorimotor sequence learning in cortex - Identical network of pyramidal cells can perform a very different computation - Difference: motion related context sent to active dendrites - Detailed list of experimentally testable properties 24 Summary
  • 25. Open Issues / Discussion Are active dendrites necessary? (Yes!) - Is a two layer network of uniform point neurons sufficient? (No!) How to integrate calcium spikes, BAC firing, and apical dendrites? Continuous time model of HTM, including inhibitory networks Collaborations We are always interested in hosting visiting scholars and interns. Co-authors: Jeff Hawkins, Scott Purdy, Marcus Lewis (Numenta), Yuwei Cui Contact info: sahmad@numenta.com @SubutaiAhmad