The vanishing central executive - Distributed neural mechanisms of decision-making

1. The vanishing central executive
Distributed neural mechanisms of decision-making
Paul Cisek
Summer School in Cognitive Sciences
Evolution and Function of Consciousness
July 4, 2012

2. Our question:
• When, where, how and why — since the origin of life on Earth
about 4 billion years ago — did organisms' input/output
functions become conscious input/output functions?
But first, another question:
• Why “input/output” functions?

3. What is behavior?
sensory motor
Perception Cognition Action
input output
representation representation
of the world of the motor plan
“The whole neural organism, it will be remembered, is,
physiologically considered, but a machine for converting
stimuli into reactions” (James, 1890, p. 372).
Behavior: An analysis of the world,
followed by deliberation and planning,
followed by execution of the plan. William James
“sense, think, act”

4. Psychological architecture for behavior
sensory motor
Perception Cognition Action
input output
representation representation
of the world of the motor plan
• University courses
• Textbooks Q: From where does this view originate?
• Journals
• Conferences
• Academic departments
• Grant review committees
• Scientists
• Questions we ask
• Theories we propose

5. “Dualism”
sensory motor
Perception Mind Action
input output
Socrates
Descartes
• Philosophy: The mind is non-physical
– This forces interfaces between non-
physical mind and physical world
• Psychology: Study of the psyche
– Structuralism: The mind is studied
through introspection John Locke
Wilhelm Wundt

6. Behaviorism
sensory motor
Perception Mind Action
input output
– Stop this metaphysical nonsense…
John Watson

7. Behaviorism
sensory motor
Perception Action
input output
– Stop this metaphysical nonsense…
– Perception and Action are directly linked
– Subject matter: Learning laws which establish the linkage
John Watson Ivan Pavlov Edward Thorndike B.F. Skinner

8. Cognitivism
sensory motor
Perception Action
input output
– Internal processes are indispensable Tolman

9. Cognitivism
sensory motor
Perception Cognition Action
input output
– Internal processes are indispensable Tolman
Shannon
– Cognition takes the mind’s place
– A fully physical process – but what
kind?
– “Information processing”
• Definition of “information”
• Definition of “processing”
– Cognition is a computational process
Turing
• Linguistics
Chomsky
• Language of thought
Fodor

10. Computational view of the brain
• The “computer metaphor”
– Cognition is like computation:
Rule-based manipulation of
representations
(Newell & Simon, Pylyshyn)
– The mind is the software (Block)
Newell & Simon
– Studies of mental phenomena may be conducted independently of
studies of brain physiology
• Less to worry about
• Not so much known (yet) about the brain
• Historical separation between psychology and biology

11. What kinds of representations?
• “Descriptive” representations
– Capture knowledge about the world
and the organism
– Explicit
– Objective, accurate to external Descriptive representations
reality, uncontaminated by internal delineate the conceptual borders
states input/output
– Examples: between the processes that
•
^
Reconstructed visual image
•
construct them and the
3-D map of the world
input/output
• Labeled objects processes that use them.
• Desired path of the hand in space ^
David Marr

12. Can we use this to understand the brain?
• Cognitive Neuroscience
– How are psychological /
cognitive functions produced by
the brain?
– Ex: Decision-making
– Based on the concepts of
cognitivism
• Computation, descriptive
representations, working
memory, attentional filters, Michael Gazzaniga
motor programs, etc.

13. Where is the central representation?
• The visual system
– Two visual processing streams:
• ventral “what”
• where
dorsal “where”
– Separate regions analyze color,
motion, form, etc.
– Separate regions for near and
far space
what
• Binding problem
– How to create the unified
representation of the world that
is needed as input for cognition?

14. Perception, Cognition, & Action Systems?
• Primary sensory and motor regions
• “Association” regions
– Appear to first encode sensory,
then motor representations Cognition?
Cognition
– Even true at the level of
Cognition
individual neurons
Example: Lateral intraparietal area
– is it “attention”?
(input to cognition)
– or “intention”?
(output of cognition)
– How could it be both?
– Could it be cognition?

15. Where is the central executive?
• Decision-making
– Neural correlates in prefrontal
and orbitofrontal cortex
– also in parietal cortex
– Premotor cortex
– Supplemental motor area
– Frontal eye fields
– Basal ganglia
– Even the superior colliculus
– Activity reflects decision everywhere at
about the same time (~150ms)

16. Conceptual challenges
• The binding problem
– How to create the unified representation of the world that is needed
as input for cognition?
• The problem of meaning
– How does a computational process know the meaning of the
representations that it manipulates?
– “Chinese Room” (Searle)
– The “symbol grounding problem” (Harnad)
– Representations are purely syntactic, they have no intrinsic
input/output
semantics, no meaning to the system that uses them
^

17. Psychological architecture for behavior
sensory motor
Perception Cognition Action
input output
• Some observations:
1. This model inherits its structure from
mind-body dualism
2. Was designed to explain the abstract
problem-solving behavior of adult humans
3. Its concepts were developed under the
explicit assumption that the substrate
doesn’t matter
• Perhaps it should not be surprising that this model has difficulty
explaining neural data…

18. Evolution
• Two key concepts:
– Natural selection
• What is the selective advantage of X?
Darwin
– Descent with modification
• What are the phylogenetic origins of X?

19. The long history of behavior

20. Are brains input/output devices?
• What else could they be?

21. What kinds of devices are living systems?
• Control systems:
– Ex: Biochemistry
• Suppose there is some substance A necessary A
for survival
• Suppose there’s a catalyst for creating A whose
action is regulated inversely by the concentration of A
• Feedback control system
• Exploits consistencies in the laws of chemistry
• Control loop within the organism: “Physiology”

22. What kinds of devices are living systems?
• Control systems can extend beyond the skin
– Ex: Kinesis
B
• Suppose substance B cannot be produced A
within the body, must be absorbed from the world
• If the local concentration of substance B falls
below desired levels, move randomly
• Exploits statistics of nutrient distributions
(assumes that there is more elsewhere) Concentration of [B]
• Control loop that extends outside the skin: “Behavior”
– Reliable motor-sensory contingencies exist
• Statistics of food distributions (move → find food)
• Laws of optics and mechanics (contract muscle → arm moves)
• Laws of interaction (you show teeth → I back off)
• Animals are constantly doing whatever brings them to the most
desirable situation (full stomach, safety, etc.)
• “Behavior: The control of perception” (Powers, 1973)

23. Different ways of looking at behavior
1. Given a perception, produce the best action
“The whole neural organism, it will be
remembered, is, physiologically considered,
but a machine for converting stimuli into
reactions” (James, 1890).
William James
2. Of the possible actions, produce that which
results in the best perception
“What we have is a circuit… the motor
response determines the stimulus, just as
truly as sensory stimulus determines
movement” (Dewey, 1896).
John Dewey

24. Ethology
• Studies of animal behavior in Von Uexküll
the wild
• Species-specific behavioral
niches
• “Closed-loop” sensorimotor
control
• Key stimuli
Lorenz & Von Holst
Tinbergen

25. What kinds of representations?
• “Descriptive” representations • “Pragmatic” representations
– Capture knowledge about the world – Used to guide interaction between
and the organism the world and the organism
– Explicit – Implicit
– Objective, accurate to external – Subjective, mix external reality and
reality, uncontaminated by internal internal state, often correlate with
many variables at once
states
– Examples:
– Examples:
• Salience map
• Reconstructed visual image • Motor signals to the limb
• 3-D map of the world • Subject-dependent opportunities for
• Labeled objects action (“affordances”)
• Desired path of the hand in space
David Marr J.J. Gibson

26. Example: Decision-making
What to do?
Move the queen?
Protect the pawn?
Threaten the
knight?
How to do it?
Which grasp point?
“Selection” What trajectory?
How to avoid
obstacles?
“Specification”
• Classical model:
– First decide what to do (select) then plan the movement (specify)
– Sense, think, act

27. Decision-making in the wild
• The world presents animals with multiple opportunities for action (“affordances”)
• Cannot perform all actions at the same time
• Real-time activity is constantly modifying affordances, introducing new ones, etc.

30. Action specification and selection
must occur in parallel

32. Sensorimotor contingencies influence
how selection should be done

33. Specification and selection in parallel
A population of tuned neurons
Distance
Cell activity
Direction
• Action Specification: Activation of parameter regions corresponding
to potential actions
• Action Selection: Competition between distinct regions of activity

34. What are the neural substrates?

35. attention Specification in the dorsal visual stream
– Cells sensitive to spatial visual information
(Ungerleider & Mishkin …)
– Involved in action guidance (Milner & Goodale)
– Divergence into separate sub-streams, each
specialized toward different kinds of actions
(Stein; Andersen; Colby & Goldberg; Matelli &
Luppino ...)
– An increasing influence of attentional effects,
enhancing information from particular regions of
interest (Duncan & Desimone; Posner & Gilbert;
Treue; Boynton ...)
– Parietal representation of external world is
“sparse” (Goldberg)

36. potential actions
attention
Fronto-parietal system
– Activity related to potential motor
actions (Andersen; Georgopoulos;
Kalaska; Wise; Hoshi & Tanji)
– Competition between potential actions
– Various biasing factors
• attention (Goldberg; Steinmetz)
• behavioral relevance
(Mountcastle; Seal & Gross)
• probability (Glimcher; Shadlen)
• reward (Glimcher; Olson)

37. potential actions
attention
behavioral
biasing
Basal ganglia
– Cortico-striatal-pallido-thalamo-cortical
loops (Alexander; Middleton & Strick)
– Selection of actions from among
alternatives (Mink; Redgrave et al.)
– Reward (Hikosaka; Schultz)

38. potential actions
attention
cognitive
decision-making
behavioral
biasing
Prefrontal cortex
– High-level decisions
based on knowledge
object about object identity
identity (Fuster; Miller; Tanji…)
– Receives ventral stream
information on object
identity (Sakata…)

39. potential actions
attention
cognitive
decision-making
predicted behavioral
feedback biasing
Execution
– Fast visual feedback
(Prablanc; Desmurget)
object – Forward models
identity
(Ito; Wolpert; Miall)
visual feedback motor
command

40. potential actions
attention
cognitive
decision-making
predicted behavioral
feedback biasing
object
identity
visual feedback motor
command

41. “Affordance competition hypothesis”
potential actions
Cisek (2007) Phil.Trans.Royal Soc. B.
•attention
Continuous specification of currently available potential actions
cognitive
decision-making
• Competition between potential action representations in fronto-
parietal regions
• Biasing from frontal and subcortical areas
predicted behavioral
feedback biasing
• Decision is made through a “distributed consensus”
object
identity
visual feedback motor
command

42. Behavior
Perception Cognition Action
audition attention propositional
proprioception vision logic forward
inverse
decision planning models
kinematics
making
object vision
trajectory action
recognition of space reinforcement
generation sequencing
learning
Behavior
Action Action
specification selection
decision
grasping reaching running attention making affect
reinforcement
forward vision proprio- inverse arm key stimulus
action learning
kinematics detection
arm models of nearby ception sequencing
space
object
propositional
recognition
logic

43. Predictions
• Multiple potential actions can be specified simultaneously
• Biased competition between potential actions
• Everything occurs in parallel

44. Neural activity specifies multiple actions
Classic model:
– Store information, decide,
then plan one action
Affordance competition:
– Specify both actions,
then select one
Time
Ce
Ce
Primary
llll P
PD
Motor
D
Cortex
Caudal
PMd
Rostral
PMd
Cisek & Kalaska (2005) Neuron

45. Predictions
• Multiple potential actions can be specified simultaneously
• Biased competition between potential actions
• Everything occurs in parallel

46. Biased choice task
1-TARGET 1 drop Alexandre
CHT 2 drops Pastor-Bernier
DELAY
3 drops
GO
THT
Reward: 1
GO
2-TARGET
FREE THT
CHT
Reward: 3
DELAY 67%
GO
THT
33%
FORCED Reward: 1

47. Neural activity in premotor cortex
• No effect of value
in 1T task
• However, if another
target is present, then
activity increases with
value of preferred
target
• If value of preferred
target is constant,
activity decreases with
value of other target
• Activity decreases with
distance between
targets
Pastor-Bernier & Cisek (2011) J. Neurosci.

48. Distance-dependent interactions
• More activity when targets are
closer
• Compare the strength of the
competition as a function of
target distance
– As distance increases, slope is
increasingly negative
• The competition is strongest
between cells with the largest
difference in preferred directions

49. Why should it matter that distance matters?
• The distance effect suggests that the decision is made
within the sensorimotor system
– If decisions were purely cognitive (“I prefer to get 3 drops of juice
over 1 drop”), then they should be determined in an abstract space
– The dynamics of the competition which determines choice depend
on the spatial relationship between the movements themselves

50. Predictions
• Multiple potential actions can be specified simultaneously
• Biased competition between potential actions
• Everything occurs in parallel

51. potential actions
attention
cognitive
decision-making
predicted behavioral
feedback biasing
object
identity
visual feedback motor
command

52. Timing
potential actions
• An animal is constantly interacting with the world
attention
– Continuous sensorimotor control of ongoing actions cognitive
decision-making
– Continuous specification of alternative actions
– Continuous evaluation of value
– Continuous tradeoffs between persisting in a given activity or
switching to a different, currently available one
predicted behavioral
feedback biasing
• Specification and selection must normally occur in parallel
• However, if we put the animal in the lab
– Time is broken into discrete “trials”object of which begins with a
each
stimulus and ends with a responseidentity
– The stimulus is deliberately made independent from the response
• What should we see? motor
visual feedback
command

53. potential actions
attention
cognitive
decision-making
predicted behavioral
feedback biasing
object
identity
visual feedback motor
command

54. potential actions
Wave 1. Fast feedforward sweep

55. potential actions
attention
cognitive
decision-making
behavioral
biasing
object
identity
Wave 2. Attentional/Decisional modulation

56. Two waves of activity
Ledberg et al. (2007) Cerebral Cortex
• Measured LFPs from various regions of cerebral cortex
• Monkeys performed a conditional GO / NOGO task

57. Two waves of activity
1. Fast feedforward sweep
• Activation in ~50ms
throughout dorsal
stream and frontal
cortex
2. Attentional/Decisional
• About 150ms post-
stimulus,
discrimination of
Go/Nogo throughout
the cortex

58. Summary 1: Experimental data
• Simultaneous specification of multiple potential actions
– Arm reaching system (PMd, PRR, M1)
– Grasping system (AIP, PMv)
– Saccade system (LIP, FEF, Superior colliculus)
• Biased competition
– Potential actions compete against each other within sensorimotor
maps, influenced by a variety of biasing factors (e.g. reward)
– NOTE: Similar mechanism as attention (Duncan & Desimone)
• “Attention” is selection near sensors, “decision” is selection near
effectors
– Influences depend on geometry – decisions are not simply abstract
• These are “pragmatic” representations, not “descriptive”
– Decision is made through a “distributed consensus”
• Parallel specification and selection systems

59. Summary 2: Theoretical concepts
• “Affordance competition hypothesis”
– Instead of serial Perception, Cognition, & Action modules, we have parallel
specification and selection systems
– Better match to neural data
– Better suited to the kinds of tasks that dominated animal behavior
• “Pragmatic representations”
– Neural activity aimed not at describing the world, but at mediating
interaction with the world
– Correlation with external and internal variables is necessary, but mixtures
are useful (e.g. spatial direction mixed with reward values)
– Conjecture: Most, but not all, neural activity is of this kind
• “Descriptive” representations (e.g. in the ventral stream) emerged in evolution as
specializations of pragmatic representations for advanced selection
• Cognitive advances evolved through hierarchical elaboration
– Diversification of fronto-parietal loops, cortico-striatal circuits, cortico-
cerebellar circuits, into progressively anterior/abstract systems
– Interaction lays the foundation for cognition (Piaget)

60. Summary 3: Philosophical implications
• There is no central executive
– Decisions emerge through a distributed consensus
• Classic problems in a different context
– Binding problem:
• Activity of separate streams is coherent by virtue of dealing with the same world
– Symbol grounding problem:
• Interaction has meaning by virtue of influencing the variables critical for life
• Symbols are specializations (“shorthand notation”) that emerged late in evolution,
already within the context of grounded interaction
– The “Hard” problem
• Feeling is different than doing
– Being inside the loop is different than observing it from the outside
– Private language, beetle in box, squirrel in head, 1 st person perspective, the “Umwelt”
• The computer metaphor
– With all due respect to Alan Turing, the computer metaphor is misleading as
a model for the brain
– What matters is control (Wiener, Ashby, Powers, Gibson, Dewey, etc.)

61. “The great end of life is not
knowledge but action”
– T. H. Huxley
(1825-1895)
“Your head is there to move
you around”
– R.E.M. (1980-2011)

62. THANK YOU
• Lab members
– Marie-Claude Labonté
– Alexandre Pastor-Bernier
– David Thura
– Ignasi Cos
– Matthew Carland
– Jessica Trung
• Alumni
– Jean-Philippe Thivierge
– Thomas Michelet
– Valeriya Gritsenko
EJLB
THE
FOUNDATION
paul.cisek@gmail.com