1. Introduction to neuroimaging:
neurons, images, inference
Geoffrey K Aguirre, MD, PhD
cfn.upenn.edu/aguirre
Center for Neuroscience & Society
Monday, March 8, 2010

2. neural activity functional MRI
image
GK Aguirre
Monday, March 8, 2010

3. raw data over time processed result
GK Aguirre
Monday, March 8, 2010

4. lying
processed result inference
GK Aguirre
Monday, March 8, 2010

5. neural activity
GK Aguirre
Monday, March 8, 2010

6. GK Aguirre
Monday, March 8, 2010

7. x-ray human head
GK Aguirre
Monday, March 8, 2010

8. CT scanner
Griff Wason
Monday, March 8, 2010

9. axial CT scan of the brain
GK Aguirre
Monday, March 8, 2010

10. Allan M. Cormack Godfrey Hounsfield
1979 Nobel Prize in Physiology or Medicine
Griff Wason
Monday, March 8, 2010

11. inject radio-labeled sugar or water
GK Aguirre
Monday, March 8, 2010

12. co-incidence detection for positron emission tomography
GK Aguirre
Monday, March 8, 2010

13. PET scan of the brain
GK Aguirre
Monday, March 8, 2010

14. GK Aguirre
Monday, March 8, 2010

15. GK Aguirre
Monday, March 8, 2010

16. B0
M0
B0
H2O
Monday, March 8, 2010

17. 90° radio-
frequency
Z pulse
B0 X
Y
high energy state
Monday, March 8, 2010

18. T1 signal RF energy
released
Z Z Z
B0 X X X
Y Y Y
high energy state protons relax low energy state
RF energy
T1 time constant
of decay
time
Monday, March 8, 2010

19. T1 weighted T2 weighted
GK Aguirre
Monday, March 8, 2010

20. FLAIR pulse sequence
GK Aguirre
Monday, March 8, 2010

21. Paul Lauterbur Sir Peter Mansfield
2003 Nobel Prize in Physiology or Medicine
Griff Wason
Monday, March 8, 2010

22. diffusion weighted (DWI) gradient echo
FLAIR MRA
GK Aguirre
Monday, March 8, 2010

23. susceptibility artifact from iron oxide particles
suspended in beeswax in hair
GK Aguirre
Monday, March 8, 2010

24. hemoglobin contains iron atoms
GK Aguirre
Monday, March 8, 2010

25. Linus Pauling paramagnetic diamagnetic
GK Aguirre
Monday, March 8, 2010

26. Deoxyhemoglobin creates
magnetic field inhomo-
geneities, causing spin
dephasing
Z
X
Y
protons de-phase
Monday, March 8, 2010

27. gradient echoplanar image
GK Aguirre
Monday, March 8, 2010

28. Neuro-vascular coupling
↑ blood flow
physiologic delay ↑ blood volume
↓ [deoxy-Hgb]
Monday, March 8, 2010

29. T2*
imaging signal
neuro-imaging
coupling
time(secs)
GK Aguirre
Monday, March 8, 2010

30. raw data over time processed result
GK Aguirre
Monday, March 8, 2010

31. voxels
3x3x3 mm
~ 10 million neurons
GK Aguirre
Monday, March 8, 2010

32. time
neural activity
fMRI signal
1 2 3 4 5 6 7 8 9
time (seconds)
Monday, March 8, 2010

33. A hemodynamic proxy of neural activity
T2*
imaging signal
neuro-imaging
coupling
time(secs)
GK Aguirre
Monday, March 8, 2010

34. A nearly linear system
T2*
low-pass
~ linear
system
neural activity BOLD fMRI signal
GK Aguirre
Monday, March 8, 2010

35. The BOLD fMRI system in space
Spatial resolution of about
0.025 cm3
In a typical 3 x 3 x 3 mm
voxel, there are ~ 10 million
neurons
Monday, March 8, 2010

36. 1
imaging signal
0.5
0
-0.5
0 2 4 6 8 10 12 14
time(secs)
Temporal resolution of 3-4 s Spatial resolution of ~0.025 cm3
(3x3x3 mm, ~10 million neurons)
Monday, March 8, 2010

37. Compare conditions over time
BOLD fMRI signal
+ + +
GK Aguirre
Monday, March 8, 2010

38. Color-coded statistical result:
BOLD fMRI signal
t-value = 2.1
0.0 7.0
t-statistic
0.0 -7.0
GK Aguirre
Monday, March 8, 2010

39. Neuroaesthetics
thresholding digital hi-res surface
smoothing anatomical reconstruction
GK Aguirre
Monday, March 8, 2010

40. A relative signal
The BOLD fMRI signal has no
direct, absolute interpretation.
Must be compared between states
studied close together in time.
Monday, March 8, 2010

41. Neural activity?
A hemodynamic proxy of neural activity
younger
• fMRI measures blood flow;
imaging signal
differences in vascular response older
can confound differences in
neural activity
time(secs)
drugs, hormonal states, age, gender
GK Aguirre
Monday, March 8, 2010

42. A generalizable result?
Studied population
healthy college middle-aged children
students lawyers
? ?
GK Aguirre
Monday, March 8, 2010

43. An integrated measure
Detectable
• a bulk change in neural activity
Undetectable
• a change in population code
The BOLD fMRI signal integrates neural
activity over seconds and millimeters.
(spike rate vs. local field potential)
Monday, March 8, 2010

44. raw data over time processed result
GK Aguirre
Monday, March 8, 2010

45. love
processed result inference
GK Aguirre
Monday, March 8, 2010

46. Three basic types of neuroimaging
studies
The key question to ask for each
GK Aguirre
Monday, March 8, 2010

47. cognition / consciousness physics / physiology
working
memory
T2*
4
system #1 system #2
..
GK Aguirre
Monday, March 8, 2010

48. Three basic types of neuroimaging
studies
The key question to ask for each
GK Aguirre
Monday, March 8, 2010

49. The brain area for “Love”?
Forward inference
Love
?
which brain areas correspond to
an isolated behavior? GK Aguirre
Monday, March 8, 2010

50. The brain area for “Love”?
Forward inference
Spouse - Friend
= Love
isolate behavior by subtracting
conditions GK Aguirre
Monday, March 8, 2010

51. The brain area for “Love”?
Spouse vs. Friend over time
GK Aguirre
Monday, March 8, 2010

52. The brain area for “Love”?
Forward inference
= Love
= Familiarity?
= Obligation?
= Sexuality?
what if the “subtraction” includes
other mental states? GK Aguirre
Monday, March 8, 2010

53. The brain area for “Love”?
Forward inference
brain activity
love?
Love amount of love
LOVE!
relate variations in the behavior
to variations in neural response GK Aguirre
Monday, March 8, 2010

54. The brain area for “Love”?
Forward inference
brain activity
amount of love
love?
Love
LOVE!
relate individual differences in
behavior to brain differences GK Aguirre
Monday, March 8, 2010

55. Forward inference
how was the behavior isolated?
GK Aguirre
Monday, March 8, 2010

56. Clinton induces “Conflict”?
Focal reverse inference
Conflict
conflict
use local brain activity to identify mental
states or emotions a situation evokes GK Aguirre
Monday, March 8, 2010

57. • Why we love
• What your brain looks like on faith
• What makes us moral
• When worry hijacks the brain
• How we get addicted
Scratching evokes amake us
People with complicated grief
Untrue statements sense of
• Marketing to your mind
feel pleasurejust like seeing
experiencebecause it
disgust, paradoxical
decreasesduring sadness
pleasure memory of pain
rotten food
• Inside the grieving brain
• It feels good and everybody does it [scratching]
• Mind reading is now possible
• This is your brain on optimism
• Hot flashes [fMRI of menopause]
GK Aguirre
Monday, March 8, 2010

58. Clinton induces “Conflict”?
Focal reverse inference
Conflict
conflict
use local brain activity to identify mental
states or emotions a situation evokes GK Aguirre
Monday, March 8, 2010

59. Clinton induces “Conflict”?
Focal reverse inference
Conflict
unpleasant
memories
?
hope
decision
making
what if more than one emotion can
activate a brain region? GK Aguirre
Monday, March 8, 2010

60. “Boring neuroscience”
Implementation
brain activity
speed of
motion
relate variations in stimulus properties or
information processing to response GK Aguirre
Monday, March 8, 2010

61. Focal reverse inference
how strong is the association between local brain
activity and the assumed evoked behavior?
GK Aguirre
Monday, March 8, 2010

62. Are you lying?
Distributed reverse inference
I picked the
ace of spades!
measure distributed patterns of response
to classify mental states GK Aguirre
Monday, March 8, 2010

63. • Why we love
• What your brain looks like on faith
• What makes us moral
• When worry hijacks the brain
• How we get addicted
fMRI scanner can determine if
The dataof 10,000 different
Which can determine what
• Marketing to your mind
tool you are currently thinking
pictures you are viewingthe
you are lying or telling can
about (hammer or wrench)
be read from your cortex
truth
• Inside the grieving brain
• It feels good and everybody does it [scratching]
• Mind reading is now possible
• This is your brain on optimism
• Hot flashes [fMRI of menopause]
GK Aguirre
Monday, March 8, 2010

64. Are you lying?
Distributed reverse inference
Iʼll tell the truth
1) train a computer to learn the pattern of
activity seen with different mental states GK Aguirre
Monday, March 8, 2010

65. Are you lying?
Distributed reverse inference
Iʼll lie and say I
have a jack
1) train a computer to learn the pattern of
activity seen with different mental states GK Aguirre
Monday, March 8, 2010

66. Are you lying?
Distributed reverse inference
?
I picked the
three of clubs!
2) classify an unknown brain pattern as
belonging to a particular state GK Aguirre
Monday, March 8, 2010

67. Are you lying?
Distributed reverse inference
?
I never killed a man
in Reno just to watch
him die!
How stimulus or context dependent is the
effect? Particularly relevant for lie detection... Aguirre
GK
Monday, March 8, 2010

68. Multi-voxel classification
can the classification be generalized beyond the
training context?
GK Aguirre
Monday, March 8, 2010

69. Forward inference
determine which brain region is associated with an
isolated behavior
Focal reverse inference
use localized brain activity to determine which
mental states are evoked by a complex behavior
Multi-voxel classification
use distributed patterns of brain activity to predict
which mental state is being experienced
GK Aguirre
Monday, March 8, 2010

70. Forward inference
how was the behavior isolated?
Focal reverse inference
how strong is the association between local brain
activity and the assumed evoked behavior?
Multi-voxel classification
can the classification be generalized beyond the
training context?
GK Aguirre
Monday, March 8, 2010

71. Monday, March 8, 2010