Presentation for International Narrative 2016 conference on proposed work bringing together narrative, semiotics, and cognitive neuroscience to study game narrative.
Cognitive intersections: Meeting Narrative, Semiotics, and Neuroscience in Video Game Characters
1. Cognitive Intersections: Meeting
Narrative, Semiotics, and Neuroscience in
Video Game Characters
Cody Mejeur
Department of English
Michigan State University
mejeurco@msu.edu
@cmejeur
2. A Question… Or Two.
What is the relationship between the cognitive and semiotics?
Addition: What is the relationship between the cognitive,
semiotics, and narrative?
Focus: Video Game Characters
3. Semiotics and Narrative
Signs in games (including characters) come
together
Play between them is narrative
Question of scale?
Analysis of narrative (particularly close
reading/analysis) requires focus on particular
signs and groups of signs
Extract out to larger structures
Always scripted and emergent qualities
6. The Problem of Structure
Does structure actually match experience?
What do structures miss or exclude?
Always building yet another model?
What would narrative look like without structure?
Narrative as less of a structure, system,
taxonomy, etc., more of a force or cognitive
process? Less what it is, more what it does?
Can we design structures that are more fluid,
dynamic, adaptable, experiential?
A Thought Experiment
8. Semiotics and Cognition
Cognitive Semiotics and Cognitive Linguistics
Focus on neuroscience of individual letters,
sounds, words
Expanding to compound words, clauses,
phrases, sentences.
How does player use/interact with it?
Communication?
Constructing a Narrative?
Do signs accumulate?
Change over time according to scripted and
emergent relations
10. Signs and Players
All signs in games, especially characters, have
some relationship to the player
Signs are never fully stable
Always moving, changing
Limits to change, sign must cohere
Force relations within and between signs
11. Speaking of Player Perception… Narrative and
Cognition
Signs coming together in narrative
Difficulty of studying narrative via neuroscience
Assumption that activity or interface=narrative?
Challenging the Narrative/Play Distinction
Theory?
Jenkins, Ryan, Walsh, Prince
Degrees of Narrativity, Embedded/Emergent/Experiential
Narrative
Social Scientific Methods?
Interview players
Forums research
Problem of sample sizes-is this inevitable?
12. Designing a fMRI experiment
Test distinction/relationship between
narrative and play
Problem: Controls, Isolating
“Narrative”
Choice: Use First Person games
(shooters, exploration, etc.)
Avoid character entanglements
Mirror/Mimic Player
POV/Consciousness
Question of interface
Alexander Galloway–what
assumptions are built into the
interface?
What are the narrative and cognitive
implications of the interface?
13. Screening Questionnaire
Control for experience, predisposition toward or against
narrative
fMRI Play
MRI compatible controller
Eye-tracking, video recording
Play segments of three games: Halo, Half-Life 2,
Bioshock. Similar play, different narrative
configurations.
Post Scanner Questionnaire and Oral Interview
Modified Game Experience Questionnaire, Eindhoven
University of Technology
Steps to Experiment
14. Final Suggestions/Questions
Bringing together Semiotics, Narrative, and Cognition
(inc. Cognitive Neuroscience) can help us understand
video game characters and narrative meaning-making
in games
The reverse is also true: games can change our
concepts and theories
Challenge us to rethink narrative and our reliance
on structure
To what extent is this possible?
What are the benefits and limitations of neuroscience?
What does the study of game narrative look like when
it moves past model-making?
