prepared for neuroeconomics book study chapter 6. by Sanfey and Dorris

1. Games in Humans and Non-human Primates:
Scanners to Single Units
The prospects for game theoretical approach to
neuroscience of decision making
Prepared for
Neuroeconomics book study
Chapter 6. by Sanfey and Dorris
2009.04.07, 07p.m.
Kyongsik Yun, Ph.D. Candidate
KAIST
yunks@kaist.edu

2. From matrix to real world
The gap between the experiments and
real world in decision making studies
• Traditional decision making studies were done in the
confined experimental settings, isolating a specific
aspect of complex decision making (e.g. risky,
uncertain, ambiguous).
• However, real life decisions are more complex and
made in the context of a social interaction.

3. The combination of game theoretical tasks and
modern neuroscientific methods is the solution.
• This chapter examines
– Game theory
– Invasive electrophysiological techniques in
monkeys during game-theoretic tasks
– Non-invasive imaging techniques in humans
during game-theoretic tasks

4. Game theory attempts to mathematically capture behavior in
strategic situations, in which an individual's success in
making choices depends on the choices of others.
– von Neumann and Morgenstern (1947) Utility Theory
Utility theory is unable to explain why
people are often simultaneously attracted
to both insurance and gambling.
• Prospect theory
– (Kahneman & Tversky 1979)
Real world decision making is less selfish and more willing to consider
reciprocity and equity (Camerer 2003).

5. Ultimatum Game
Player 1 can offer a fair (F) or unfair
(U) proposal; player 2 can accept (A)
or reject (R).
Guth et al. 1982
Nash Equilibrium is at odds: Low offers of less than 20% of the total amount
are rejected about half of the time.
Neuroscience has begun to offer clues as to the mechanisms underlying these
decisions.

6. Trust Game
48 = 12 MU own endowment + 36 MU tripled transfer
A rational and selfish trustee will never honor the trust given
by the investor. In real situation, a majority of investors send
some amount of their money to the trustee.
Baumgartner et al. Neuron 2008

7. Prisoner’s dilemma game-
simultaneous trust game
• The Nash Equilibrium is mutual defection.
• Players exhibit 50% mutual cooperation.
Cooperate defect
Cooperate
defect

8. Public goods game: a generalized
form of PDG
• The self-interested solution is to free-ride
and hope that everyone else contributes.
• Players on average contribute about half
of their endowment to the public good.

9. Mixed strategy game
(e.g. matching pennies and the inspection game)
• A pure strategy provides a complete
definition of how a player will play a game.
• A mixed strategy is an assignment of a
probability to each pure strategy.
• The Nash Equilibrium is to select randomly
with equal probabilities.
• Players attempt to infer the strategy of our
opponent, using theory-of-mind processes.

10. It is unclear whether the decisions emerge
from strategic or altruistic motivations.
• Ultimatum Game proposer behavior
– fairness? Fear to be rejected?
• Examining these games in a neural
context can begin to offer clues as to the
motivations behind the decisions.

11. The benefits of combined approach between
experimental economics and neuroscience methods
• More precise characterizations of
behavior (how decisions are actually
made)
• Neuroscience can provide important
biological constraints on the processes
involved.

12. The animal model
(Rhesus Macaque)
• To investigate “black box” during social
interactions
• For studying higher-order decision
processes
• General organization of their nervous
system is similar to that of humans
• Monkeys and humans display
comparable strategies in mixed-strategy
games

13. Advantages and disadvantages of a systems
neurophysiology (invasive animal model) approach
• Advantage:
– direct access to the neural substrate
– Exquisite temporal (<1ms) and spatial (ind. neurons) resolution
– Artificial manipulation of neuronal activity can provide causal
evidence, complementing the correlational evidence provided
by neuronal recordings.
• Disadvantages
– Limitations exist in using non-human primates to infer the
neural processes underlying human social interactions
– Not suitable for more sophisticated games (UG, PDG)
– No verbal instruction and using only operant conditioning
techniques

14. Saccadic ( jumping-fixating) eye
movements
• A saccade is a fast movement of an eye,
head or other part of an animal's body
or device. It can also be a fast shift in
frequency of an emitted signal or other
quick change.

15. Benefits of visuosaccadic system
investigation for decision making
• Efficiently extract visual information in
perceptual decision making
• The neural circuitry underlying visual
processing and saccadic control is well
understood.

16. Adapting games for non-human primates
LIP: estimating the desirability of sensory stimuli
SC: selecting/preparing actions, (control eye movement)
PFC: evaluating the consequences of actions
LIP – gateway for the
emergence of the brain
Newsome and Shadlen 2001 Nature

17. Encoding the desirability of choice
stimuli in LIP
• At the end of visual
processing stream
• Encode the saliency of
visual targets
Platt and Glimcher 1999 Nature
Sugrue, Corrado, Newsome 2005 NRN

18. Visuosaccadic version of mixed-
strategy inspection game
Nash Eq.
0.9 0.3
0.5
Inspection cost
Dorris and Glimcher 2004 Neuron

19. LIP firing rates varies with desirability,
not probability of outcome
Dorris and Glimcher 2004 Neuron

20. Correlation between LIP Firing Rates and an
Estimate of the Relative Desirability of Choices
Dorris and Glimcher 2004 Neuron

21. Evolving response selection in
midbrain superior colliculus
• Intimately involved in saccade generation
• See Figure 6.4 (Dorris unpublished)
– B) Preferred (black)/ unpreferred (red)
– C) Stimulus induced saccades deviate
slightly towards the target.
– D) angular deviation of saccades increases
as the time of target presentation
approaches. (일찍 자극, 적게 변화)

22. Evaluating the consequences of
actions in frontal cortex
• dlPFC is involved in integrating choice
(i.e. left vs right) and reward (i.e.
rewarded or unrewarded) information.
• dACC encoded critical information about
the temporal delay of previous rewards
within a sequence of responses (Seo and
Lee 2007)

23. General significance of decision making studies in
non-human primates
• Recording single neurons allow for
moment-to-moment correlations
between neuronal activity and behavioral
responses
• Providing unprecedented insight into the
neuronal mechanisms underlying
stochastic choice.

24. Games in humans
• Imaging studies of social interaction
– fMRI, PET
– Stimuli of other human faces (static, 2D)
(Winston et al. 2002)
– Mentalizing (Gallagher et al. 2000)
– Moral reasoning (Greene et al. 2001)
Pioneering works of its kind

25. Automatic and intentional brain responses
during evaluation of trustworthiness of faces
Explicit judgments: whether an individual was trustworthy
Implicit judgments: an unrelated age assessment
Insula
STS
l. Amyg r. Amyg.
The findings extend a proposed model of social cognition by highlighting a
functional dissociation between automatic engagement of amygdala versus
intentional engagement of STS in social judgment.
Winston, Strange, O’Doherty, & Dolan NN 2002

26. Reading the mind in cartoons and stories
medial prefrontal cortex (paracingulate cortex)
Gallagher et al. 2000

27. Problems in the previous studies
• Does the pattern of brain activation in
response to the picture of a static, 2D face
accurately reflect the brain’s response to
the dynamic, embodied faces that we
encounter in everyday life?
• Is the pattern of brain activation in
response to reasoning regarding
hypothetical, fictitious scenarios the same
as when grappling with significant real life
social problems?

28. Solutions to improve ecological validity of
experiments in neuroeconomics
• To interact with other people in real
social exchanges from outside the
scanner (McCabe et al. 2001)
• Hyperscanning technology (Montague et
al. 2002)
– Data collection efficiency
– Open new vistas in social cognitive
neuroscience

29. Another solution is to manipulate
specific neurotransmitter systems
• Tryptophan depletion can be used to decrease brain
serotonin levels, reducing cooperative behavior
(Wood et al. 2006)
• Oxytocin elevation increases trust (Kosfeld et al.
2005)
• TMS to temporarily activate or deactivate a brain
region and then examine its effects on decision
making (van’t Wout et al. 2005)
• Lesion study (Koenigs and Tranel 2007)

30. Trust Game behavior
Oxytocin increased trust.
Oxytocin: filled bars
Placebo: open bars
Baumgartner et al. Neuron 2008

31. Fool me once, shame on you.
Fool me twice, shame on oxytocin.
Baumgartner et al. Neuron 2008

32. What is the role of PFC in emotion vs.
cognition conflict decision making?
Economic rationality: Social rationality:
• economic value maximization Social value maximization
•
Fairness monitoring
•
• Absolute valuation Relative valuation
•
• vmPFC lesion -> R↑ dlPFC disruption -> A↑
•
Knoch et al. Science 2006
Koenigs & Tranel, J Neurosci. 2007

33. Current research direction:
social motivation reward
• Striatum: scale reward magnitude (Cromwell and Schultz
2003), specifically the magnitude of monetary reward or
punishment (O’Doherty 2004, Knutson and Cooper 2005)
• Striatum may register social prediction errors to guide
decisions about reciprocity (Rilling et al. 2002)
• Caudate: related to how much reciprocity the investor had
shown on previous trials, “intention to trust” (King-Casas et
al. 2005)
• caudate: altruistically punish (de Quervain et al. 2004)
• Social altruism: the striatum was engaged both by receiving
money and by donations to charity (Moll et al. 2006), and
the activation was enhanced when the donation was
voluntary as opposed to forced (Harbauch et al. 2007). ->
important implications for informing public policy

34. Caution should be used when attempting to
“reverse engineer” from patterns of brain
activity to cognitive and social processes
• Correlation analysis, TMS modulation, or patient
work is crucial to buttress the causal relationship
between behavior and brain activity.
• These results appear to demonstrate that complex
social processes recruit more basic mechanisms
within the human brain, providing support for the
notion that the brain uses a common reward metric.
This also furthers the connection between the
disparate branches of neuroeconomics from primary
and secondary rewards (food, money) to more
abstract social rewards (reciprocity, fairness).

35. Competition, Cooperation and Coordination
• Classical models of decision making,
both utility theory and game theory,
have largely ignored the influence of
emotions on how decisions are made,
but recent research has begun to
demonstrate their powerful effect.

36. The emotional brain
The Papez circuit theory (1937) of the functional neuroanatomy of emotion
Emotional experiences or feelings occur
when the cingulate cortex integrates
these signals from the hypothalamus with
information from the sensory cortex.
stream of thinking
stream of feeling
Output from the cingulate cortex to the
hippocampus (3) and then to the
hypothalamus (4) allows top–down
cortical control of emotional responses.
Dalgleish 2004 NRN

37. MacLean’s limbic system theory of the
functional neuroanatomy of emotion
the hippocampus received sensory inputs
from the outside world as well as
information from the internal bodily
environment (viscera and body wall).
Emotional experience was a
function of integrating these internal and
external information streams.
Dalgleish 2004 NRN

38. Map of brain areas commonly activated in
social decision-making studies.
Theory of Mind: STS, MPFC, OFC
Sanfey 2007 Science

39. Neuroscientific studies of emotions
• Negative emotional states have been observed behaviorally
as a result of both inequity and non-reciprocity, such as
unfair offers in a UG.
• These emotional reactions have evolved precisely to foster
mutual reciprocity, to make reputation important, and to
punish those seeking to take advantage of others (Nawak
et al. 2000)
• Neuroscienctific studies of this nature offer the potential to
go beyond speculation and to examine the causal
relationship between an emotional reaction and subsequent
social decision, as well as investigating whether areas
specialized for the processing of basic emotions may be
coopted for more complex affective reactions.

40. Behavior of social interaction
in the Ultimatum Game
Participants had a stronger emotional reaction to unfair offers
from humans than to those from computers.
40
Sanfey et al. Science, 2003

41. Neuroimaging results of the
Ultimatum Game
Sanfey et al. Science, 2003
Anterior insula: pain, distress, hunger, thirst (Denton et al. 1999), autonomic
arousal (Critchley et al. 2000)
Right anterior insula: aversive conditioning (Seymour et al. 2005)
Defection rate in PDG is correlated with right anterior insula activation (Rilling
et al. 2008)
Anterior insula may play a role in marking a social interaction as aversive, and
thus discouraging trust.

42. dlPFC is linked to cognitive processes
• Frontal, “top-down” processes in reward
studies suppress striatal activation.
• TMS increased acceptance rate of unfair
offers, providing strong evidence for a
causal relationship between activation in
this area and social decision making
(Knoch et al. 2006)
– Why social? Disruption of dlPFC using TMS
might reduce cognitive processes to accept
the offer, decreasing acceptance rates.

43. Modulating affective system in
the Ultimatum Game
• People in a sad mood (watching a 5-minute
video rated as “sad”) reject more 'unfair'
offers (Harle and Sanfey 2007).
• Examining decision making performance in
participants with disregulated emotional
processing, such as patients with
depression or schizophrenia, may be a
useful future avenue of research.
• Contradictory study: vmPFC lesion patients
also reject unfair offers more frequently
than do controls (Koenigs and Tranel 2007)

44. Significance of neuroscience
approach to decision making
• Decision making appears to involve the
interaction among multiple subsystems
governed by different parameters and
possibly even different principles.
• Neuroscience approach to decision making
has the potential to inform economic
theories of interactive decision making
(inequity aversion, social utility functions)
• Economic models based on the underlying
neural patterns may provide useful
constraint.