Presented by Mohammed Yeasin for the 2016 Commercial Drone Expo in Memphis, Tennessee

1. Unified Framework for Learning
Representation from EEG Data
Pouya Bashivan, PhD candidate
Dr. Mohammed Yeasin
The University of Memphis
April 9, 2016
1

2. Working Memory Significance
• What: Shared resource for storage and manipulation of information (e.g.
reading comprehension, arithmetic)
• Why: Plays a key role in determining individual mental capacity
• How: Information is stored and maintained through a network of cortical
and sub-cortical areas distributed widely across the brain
Meta-analysis result of 189 WM experiments
Rottschy et al. Neuroimage (2012)
2
Intro Results FutureMethods

3. Participants
• 15 participants (8 female)
• Graduate level education
• 24-33 years (28±3)
• Strongly right-handed
(Edinburgh handedness
inventory)
3
Intro Results FutureMethods

4. EEG Recording
• Inside an electro-acoustically shielded
booth
• EEG cap with 64 scalp electrodes,
standard 10-10 locations
• Electrode impedance < 5kΩ
• Sampling freq. 500 Hz
• 4 electrodes placed around
the eyes for detection of
ocular activity
4
Intro Results FutureMethods

5. Sternberg WM Task
• Two variants of Sternberg WM
task, differed by modality
• English characters
• Number of items = {2, 4, 6, 8}
• 60 trials per set size
• Visual SET presented
concurrently, Auditory SET
presented sequentially.
5
Visual
Auditory
Intro Results FutureMethods

6. Preprocessing EEG
• Down-sampling: Data down-sampled to 250 Hz
• Filtering: Band-Pass zero-phase FIR filter, N=500,
[1-45] Hz
• Epoching: Data was segmented
from 2000ms pre-SET till TEST
• Artifact Correction:
– Threshold on EOG channels
– PCA on aggregated artifactual time-series
– Discard first 3-components
6
Intro Results FutureMethods

7. Spectral properties of EEG
• Rhythmic activities of EEG = informative neuro-markers.
– Frequency-specific Power
– Frequency-specific Phase
• Conventional frequency bands:
– Delta (δ) [1-4Hz]
– Theta (θ) [4-7Hz]*
– Alpha (α) [8-13Hz]*
– Beta (β) [13-30Hz]*
– Gamma (γ) [30+Hz]
7
Intro Results FutureMethods

8. Representation Learning
• Goal: Find signal representations that are
robust to variations.
• Sources of variations:
1. Cortical maps
2. Head shapes and caps
8
Intro Results FutureMethods

9. How to Learn?
• Conventional approach: Aggregate features
into a vector
– Relationship between features is lost.
• Alternative approach: Structure features into
its natural form, preserving relationships.
– Neighborhood notion is preserved throughout
classification.
9
Intro Results FutureMethods

10. Transforming Time-series to Images
1. Project electrode locations into 2D surface
2. Compute PSD for each channel times-series
3. Compute power estimates within a specific band
4. Map power values onto the projected electrode
locations (polar projection).
5. Interpolate values to construct an image.
10
1
4
5
2, 3
Intro Results FutureMethods

11. Innovative Approach
11
Bashivan et al. ICLR (2016)
VGG ConvNet with receptive field (3x3)
Zisserman et al. ICLR (2015)
Intro Results FutureMethods

12. ConvNet
• Advantage: invariant to partial translation and deformation of
input patterns.
• Layer types:
1. Conv. Layers (filters or kernels)
2. Pooling Layers (e.g. Max(0, x))
• All parameters learned through
back-propagation
12
Conv. Layer
Max Pool
Intro Results FutureMethods

13. Handling Time - RNNs
1. Max-pooling across time
2. Temporal convolution
3. Long Short-Term Memory
(LSTM)
13
LSTM
Cell
Conveyor belt
Forget gate Input gate Output gate
Ng. et al. CVPR (2015)
Bashivan et al. ICLR (2016)
Hochreiter et al. Neural Computation (1997)
Intro Results FutureMethods

14. Behavioral Responses
• Tracking behavioral markers:
1. Response times
2. Response Correctness
14
Intro Results FutureMethods

15. Classification Results
• Adding temporal information considerably decreased average
error rates.
• Maxpooling over time frames did NOT help with classification.
• Mixture of LSTM and 1D-conv worked best.
15
Intro Results FutureMethods

16. Learned Representations
We used Deconvnet1 approach for back
projecting feature maps.
• Increasingly sparser feature activation maps
for deeper layers.
• Frequency selectivity in learned filters.
• Noticeable links to electrophysiological
markers of cognitive load (e.g. Frontal β, θ,
Parietal α)2,3
16
1. Zeiler et al. ECCV (2014)
2. Tallon-Baudry et al., Vis Neuroscience (1999)
3. Jensen et al., Eur J Neuroscience, (2002)
Intro Results FutureMethods

17. Future Work
• Brain-Computer Interface
– Real-time recognition of user intention
– Network Description Brain Activities
17
Intro Results FutureMethods

18. Questions?
18
“The only stupid question is
the question that is never asked.”
Ramon Bautista

19. Acknowledgement
• My advisers
• My friends at CVPIA lab
• 15 anonymous participants in our experiment
Bahareh
Elahian
Iftekhar
Anam
Shahinour
Alam
Dr. Mohammed
Yeasin
Dr. Gavin
Bidelman
Faruk
Ahmed
Rakib
Al-fahad
19