Does the neocortex use grid cell-like mechanisms to learn the structure of objects? by Jeff Hawkins (04/17/18)
•
4 recomendaciones•5,912 vistas
These are Jeff Hawkins' slides from the Computational Theories of the Brain Workshop held at the Simons Institute at UC Berkeley on April 17, 2018. Abstract: In this talk, I propose that the neocortex learns models of objects using the same methods that the entorhinal cortex uses to map environments. I propose that each cortical column contains cells that are equivalent to grid cells. These cells represent the location of sensor patches relative to objects in the world. As we move our sensors, the location of the sensor is paired with sensory input to learn the structure of objects. I explore the evidence for this hypothesis, propose specific cellular mechanisms that the hypothesis requires, and suggest how the hypothesis could be tested. References: “A Theory of How Columns in the Neocortex Enable Learning the Structure of the World” by Jeff Hawkins, Subutai Ahmad, YuWei Cui (2017) “Place Cells, Grid Cells, and the Brain’s Spatial Representation System” by Edvard Moser, Emilio Kropff, May-Britt Moser (2008) “Evidence for grid cells in a human memory network” by Christian Doeller, Caswell Barry, Neil Burgess (2010)
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a Does the neocortex use grid cell-like mechanisms to learn the structure of objects? by Jeff Hawkins (04/17/18)
Similar a Does the neocortex use grid cell-like mechanisms to learn the structure of objects? by Jeff Hawkins (04/17/18) (20)
Más de Numenta
Más de Numenta (15)
Último
Último (20)
Does the neocortex use grid cell-like mechanisms to learn the structure of objects? by Jeff Hawkins (04/17/18)
- 1. Computational Theories of the Brain Simons Institute April 17, 2018 Jeff Hawkins jhawkins@numenta.com Does the Neocortex Use Grid Cell-Like Mechanisms to Learn the Structure of the World? A Framework for Cortical Computation
- 2. 1) Reverse engineer the neocortex - an ambitious but realizable goal - seek biologically accurate theories - test empirically and via simulation 2) Enable technology based on cortical theory - active open source community - foundation for machine intelligence
- 4. “What is conspicuously lacking is a broad framework of ideas within which to interpret these different approaches” Francis Crick, 1979 writing about the state of neuroscience
- 6. Diversity of Function Commonality of Circuitry Regions look similar - layers of cells - vertical connections (primary) - horizontal connections - suggests columnar organization ~100 regions - vision - touch - audition - languages - all cognition Remarkably similar circuitry Vernon Mountcastle’s Big Idea: 1) All regions do the same thing, the “function” of a region is determined by its inputs. 2) Columns are the functional unit of the neocortex. (~ 150K in human) 3) Understanding the column is the key problem in neuroscience. V. Mountcastle, 1978 Hierarchical Remarkably diverse functionality
- 7. L2 L3a L3b L4 L6a L6b L6 ip L6 mp L6 bp L5 tt L5 cc Cortical Columns are Incredibly Complex L6: Zhang and Deschenes, 1997 50% L5 CTC: Guillery, 1995 Constantinople and Bruno, 2013 <10% Cortex Thalamus Cortex Motor Thalamus - 100K neurons, 500M synapses (1mm2) - Ten or more cellular layers - Dozens of intra- and inter-column connections - Inhibitory neurons/circuits are equally complex - Significant region-to-region variability Observation: The cortex is constantly predicting its input. Question: How does the cortex (column) learn predictive models of its input?
- 8. Deciphering the Cortical Column One Layer at a Time Learn predictive models of sequences +Learn predictive models of sensorimotor sequences + Grid cell-like location layer +Learn composite objects +2nd Location layer Hawkins and Ahmad, Frontiers in Neur Circ 2016/03/30 4 other papers Hawkins, Ahmad and Cui Frontiers in Neur Circ 2017/10/25 Lewis and Hawkins Poster: Cosyne 2018 Lewis and Hawkins Poster: Cosyne 2018 L2/3 L4 L5tt L6a L6b L2/3 L4 L6a L2/3 L4L4 EC-derived location
- 9. Proximal synapses: Define classic receptive field of neuron Distal synapses: Cause dendritic spikes Put the cell into a depolarized, or “predictive” state Hypothesis: Depolarized neurons fire sooner, inhibiting nearby neurons. A neuron can predict its activity in hundreds of learned contexts. 5K to 30K excitatory synapses - 10% proximal - 90% distal Distal dendrites are pattern detectors - 8-15 co-active, co-located synapses generate dendritic spike - sustained depolarization of soma HTM Neuron Prediction Starts in the Neuron Pyramidal Neuron (Major, Larkum and Schiller 2013)
- 10. Properties of Sparse Activations Example: One layer of cells 5,000 neurons, 2% (100) active Hawkins, Ahmad, 2016 Ahmad, Hawkins, 2015 1 pattern (100 active cells) Union 10 patterns (1,000 active cells) 4) Unions of patterns do not cause errors in recognition. 1) Representational capacity is virtually unlimited. (5,000 choose 100) = 3x10211 2) Randomly chosen representations have minimal overlap. 3) A neuron can robustly recognize an active pattern by forming just 8 to 20 synapses. Hypothesis: Unions are used to represent uncertainty throughout the cortex. Activity gets sparser as certainty increases.
- 11. A Input Layer Network Model for Sequence Memory No prediction Predicted input (Hawkins & Ahmad, 2016) (Cui et al, 2016) - High capacity (learns 100’s K transitions) - Learns high-order sequences: “ABCD” vs “XBCY” - Extremely robust (parameters, noise, and faults) - Learning is unsupervised, continuous, and local - Satisfies many biological constraints - Multiple open source and commercial implementations t=0 t=1 Sparse pattern = Input in specific context Next prediction t=2 t=0 t=1
- 12. Deciphering the Cortical Column One Layer at a Time Learn predictive models of sequences +Learn predictive models of sensorimotor sequences + Grid cell-like location layer +Learn composite objects +2nd Location layer Hawkins and Ahmad, Frontiers in Neur Circ 2016/03/30 4 other papers Hawkins, Ahmad and Cui Frontiers in Neur Circ 2017/10/25 Lewis and Hawkins Poster: Cosyne 2018 Lewis and Hawkins Poster: Cosyne 2018 L2/3 L4 L5tt L6a L6b L2/3 L4 L6a L2/3 L4L4 EC-derived location
- 13. Predicting Sensorimotor Sequences SensorMotor-related context How can we modify our input layer to also learn predictive models of sensorimotor sequences? Add a motor-related context. The layer can now predict its input as the sensor moves. What is the correct motor-related context?
- 15. Predicting Sensorimotor Sequences - Input layer represents “features @ locations”. - Changes with each movement. Sensed Feature Object-centric Location - “Object” layer represents object. - Stable over changing inputs. This network learns predictive models of objects. An object is “a set of features @ locations”.
- 16. FeatureFeatureFeatureLocationLocationLocation Output Input Objects Recognized By Integrating Inputs Over Time
- 17. Object Feature @ Location Location on object Column 1 Column 2 Column 3 Sensor feature Sensorimotor Inference With Multiple Columns
- 18. FeatureLocationFeatureLocationFeatureLocation Column 1 Column 2 Column 3 Output Input Recognition is Faster with Multiple Columns
- 19. Deciphering the Cortical Column One Layer at a Time Learn predictive models of sequences +Learn predictive models of sensorimotor sequences + Grid cell-like location layer +Learn composite objects +2nd Location layer Hawkins and Ahmad, Frontiers in Neur Circ 2016/03/30 4 other papers Hawkins, Ahmad and Cui Frontiers in Neur Circ 2017/10/25 Lewis and Hawkins Poster: Cosyne 2018 Lewis and Hawkins Poster: Cosyne 2018 L2/3 L4 L5tt L6a L6b L2/3 L4 L6a L2/3 L4L4 EC-derived location
- 20. Entorhinal Cortex Body in environments A B C X Y Z R S T Room 3 Room 2Room 1 Location - Encoded by grid cells - Unique to location in room AND room - Location is updated by movement A Room is: - A set of locations that are connected by movement (via path integration). - Some locations have associated features. Location - Encoded by grid-like cells in L6a - Unique to location on object AND object - Location is updated by movement Cortical Column Sensor patch relative to objects Representing Location with Grid Cells A C B Stensola, Solstad, Frøland, Moser, Moser: 2012 X Y Z W An Object is: - A set of locations that are connected by movement (via path integration). - Some locations have associated features. 1) Location representations are dimensionless. Dimensionality is defined by movement. 2) Movements do not have to be physical. They only have to exhibit path integration. 3) Features do not have to be sensory features. They can be outputs of other columns. Proposal: All knowledge, even abstract concepts, are represented this way in the cortex. Conceptual Spaces
- 21. Deciphering the Cortical Column One Layer at a Time Learn predictive models of sequences +Learn predictive models of sensorimotor sequences + Grid cell-like location layer +Learn composite objects +2nd Location layer Hawkins and Ahmad, Frontiers in Neur Circ 2016/03/30 4 other papers Hawkins, Ahmad and Cui Frontiers in Neur Circ 2017/10/25 Lewis and Hawkins Poster: Cosyne 2018 Lewis and Hawkins Poster: Cosyne 2018 L2/3 L4 L5tt L6a L6b L2/3 L4 L6a L2/3 L4L4 EC-derived location
- 22. Rethinking Hierarchy Every column learns complete models of objects. They operate in parallel. Inputs project to multiple levels at once. Columns operate at different scales of input. Sense Simple features Complex features Objects Classic Objects Objects Objects Sensor array Proposed Region 3 Region 2 Region 1
- 23. Rethinking Hierarchy Every column learns complete models of objects. They operate in parallel. Inputs project to multiple levels at once. Columns operate at different scales of input. Non-hierarchical connections allow columns to vote on shared elements such as “object” and “composite object”. Sense Simple features Complex features Objects Classic Sensor array Objects Objects Objects Sensor array vision touch Proposed Region 3 Region 2 Region 1
- 24. 1) Border ownership cells: Cells fire only if feature is present at object-centric location on object. Detected even in primary sensory areas (V1 and V2). (Zhou et al., 2000; Willford & von der Heydt, 2015) 2) Grid cell signatures in cortex: Cortical areas in humans show grid cell like signatures (fMRI and single cell recordings) Seen while subjects navigate conceptual object spaces and virtual environments. (Doeller et al., 2010; Jacobs et al. 2013; Constantinescu et al., 2016; ) 3) Sensorimotor prediction in sensory regions: Cells predict their activity before a saccade. Predictions during saccades are important for invariant object recognition. (Duhamel et al., 1992; Nakamura and Colby, 2002; Li and DiCarlo, 2008) 4) Hippocampal functionality may have been conserved in neocortex: Six-layer neocortex evolved by stacking 3-layer hippocampus and piriform cortex (Jarvis et al., 2005; Luzatti, 2015) Biological Evidence 24
- 26. Numenta Team Subutai Ahmad Marcus Lewis Thank You Scott Purdy Mirko Klukas Luiz Scheinkman