Tracking game metrics data is slowly becoming an industry standard for analyzing and improving games. Using insights from statistical analysis, games are becoming more adaptive and cater to individual experiences. Thus, biometric analysis is the latest trend to gather objective insight into player experience. Operating with game and player metric data becomes more important as game designers move from being classically rooted in the level design department to having to shift their attention towards procedural algorithms and programming that is responsible for analyzing player data. This talk will introduce the next generation of designing games based on statistical data analysis (game metrics, eye tracking and biofeedback) and discuss the challenges of these new and exciting technologies.
2. About Me
Blekinge Institute of Technology
PhD Candidate
Digital Game Development Degree
EU FUGA (“Fun of Gaming”) project
Fun & player experience research
Biometrics consulting
10. Technical quality checks
Make sure game functions correctly
Bug reports depend on skill of tester
Sometimes automatic testing
Balancing gameplay parameters
Really more a design check
Trial and error
Time-consuming
11. Classic Playtest
Watch someone play the game
Check against design intentions
Are game rules obeyed?
Are game goals reached in proper way?
Do testers report this as being “fun”?
Followed by questioning (Q&A)
Iterate design based on feedback
12. Focus groups
Modification of classic playtests
Target audience clusters
Group sessions
In-depth interviews after session
13. Think-aloud protocol
Add-on for classic playtests
Player comments playing aloud
Recorded with microphones
Spontaneous and unfiltered
Insights into player reasoning
14. Benefits of traditional
playtesting
Get a good idea of how players like
your game
Answer design questions
Watch what triggers behavior
Collect many subjective details
Uncover hidden gameplay problems
Interviews allow to investigate fine
distinctions of gameplay
15. Limitations of traditional
playtesting
Hard to generalize
Lots of bias
Observation/Memory
Testers
Questions
Subjective interpretation of behavior
Problems with accuracy
16. Why QA uses traditional
playtesting…
Works great for finding major issues
Interaction
Gameplay
Content
Interface
Uncover nuances in interviews
Insights into players’ minds
Answers to “WHY?” & “HOW?”
Direct game design feedback
17. Why QA should think about
adding next-gen testing…
Much bias in qualitative techniques
Rooted in
Analysis
Recording
Scientifically questionable
Objectivity
Reliability
Replicability
Empirical power
20. Gameplay metrics
Provide empricial insights into player
behavior
Usually event-based
Player deaths for example
Spatial data allow level design
analysis
Heatmaps
Construction of Personas
21. Example of Game Metrics
Example of game metrics data (see also Tychsen & Canossa
2008)
22. Gameplay metrics
PRO CON
Objective data Implementation
Quantifiable for specific
Identify trends
engine
Missing fine
Measure play
behavior granularity
Need statistics
Events allow
correlation with experts
biometrics Painstaking
analysis
23. Psychometric surveys
Standard psychological profiles
What motivates your players?
Standard tools from psychology
Psychotypes
Meyers-Briggs Type Indicator
EPQ-R Psychoticism
BIS/BAS Behavior
etc.
Categorize your players
24. Psychometric surveys
PRO CON
Categorize Scoring can be
players tricky
Correlate with Need statistical
personas knowledge
Validated Only fully
method valuable in
Quantifiable conjunction
Reliable
with other
measures
26. EMG
Measuring facial muscle activation
Correlates to emotions
Russel’s circumplex model of emotion
Valence = Positive or Negative
Arousal = High or Low
Brow muscle = bad mood
Smile and Eye muscle = good mood
27. Facial EMG response cumulative
means for each level
Flow Level
Immersion Level
Boredom Level
0 5 10 15
Smile (µV) Brow (µV) Eye (µV)
Objective results: Valence responses
Cumulative tests for different game level types (see also
Nacke, Lindley, 2008).
28. Correlation of Physiological Data to Events
Physiological data is recorded together with real-time game
events, allowing for automatic data clustering and analysis
29. GSR
Electrodermal activity
Eccrine sweat gland production
Two electrodes (conductance)
Correlates to arousal
Easy deployment and measurement
Signal can be noisy
Allows emotion mapping together
with EMG in circumplex model
30. Galvanic Skin Response
Cumulative Means for each level
Flow Level
Immersion Level
Boredom Level
0.86 0.88 0.9 0.92 0.94
Objective results: Arousal responses
More excitement peaks for one level (see also Nacke, Lindley,
2008).
31. Aroused
Surprise
Fear
Unpleasant Happy
Pleasant
Anger
Neutral
Disgust Calmness
Sad
Sleepiness
Not aroused
Russel’s circumplex model of emotion
The two dimensions of this model can also be mapped to EMG
and GSR measurement (see also Lang 1995).
32. EEG
Electrodes placed on scalp (from 20 to 256)
Measures electric potentials
Brainwaves are described in frequency bands
Delta (trance, sleep)
Theta (emotions, sensations)
Alpha (calm, mental work)
Low beta (focus, relaxed)
Mid beta (thinking, alert)
High beta (alert, agitated)
Gamma, seldom (information processing)
33. Game experiment Setup
EEG and EMG electrodes are being attached. The Biosemi
electrode cap consists of 32 electrodes in the areas: frontal
(F), parietal (P), temporal (T), occipital (O), central (C).
34. EEG Frequencies and Spectrum
EEG Analysis is difficult. After artifact scoring, values have to
be transformed for spectral analysis.
35. Eye Tracking
Measures what eyes look at
Saccades (fast movement)
Gaze path
Fixations (dwell times)
Attention focus
Pupil dilation/blink rate
Attention precedes gaze (200ms)
Used mainly to improve interface
Lack of 3D analysis tools
37. Example of 3D Eye Tracking Visualization
Viewed game world objects can be displayed together with
their gazepaths in 3D (see also Stellmach, 2009)
38. Physiological measures
PRO CON
Objective Expensive
Covert & Intrusive
continuous Difficult to
recording analyze
Quantifiable Time-
Reliable consuming
Replicable
Empirical power
Automatization
39. Key biometric advantages
Data is objective
Not dependent on memory/language
Continuous measurement
During event processing
Information on player responses
Emotional
Attentional/Cognitive
40. Biofeedback applications
Use fuzzy models
IEEE SIG: game.itu.dk/PSM
Player satisfaction modeling
Cognitive models
Affective models
Optimal challenge
Trigger game events with
biofeedback (e.g. Emotiv)
Popular approaches
GSR, heart-rate and respiration
42. Takeaway
1. Metrical testing is emerging
Now is the best time to jump on!
2. Your company needs user research
Ultimately your players know best!
3. Biometrics enhance classic testing
Qualitative supports quantitative data
4. Understand existing and emerging
testing methods
Keep in touch with experts
43. References
Lang, P.J. The emotion probe. Studies of motivation and attention.
American Psychologist, 50. 372-385.
R.L. Mandryk (2008). Physiological Measures for Game
Evaluation. in Game Usability: Advice from the Experts for
Advancing the Player Experience. (K. Isbister and N. Shaffer,
Eds.), Morgan Kaufmann.
Nacke, L. and Lindley, C.A., Flow and Immersion in First-Person
Shooters: Measuring the player’s gameplay experience. In
Proceedings of the 2008 Conference on Future Play: Research,
Play, Share, (Toronto, Canada, 2008), ACM, 81-88.
Tychsen, A. and Canossa, A., Defining personas in games using
metrics. In 2008 Conference on Future Play: Research, Play,
Share, (Toronto, Ontario, Canada, 2008), ACM, 73-80.
Russell, J.A. A Circumplex Model of Affect. Journal of Personality
and Social Psychology, 39 (6). 1161-1178.
Stellmach (2009). Visual Analysis of Eye Gaze Data in Virtual
Environments. Master’s Thesis.
Icons from smashingmagazine.com
44. More at Future Play!
Tomorrow, 1pm, Room 206
PANEL: Game Metrics and Biometrics:
The Future of Player Experience
Research
Featuring Mike Ambinder, Regan
Mandryk, Alessandro Canossa, Tad
Stach, and me
45. Contact Me
Lennart.Nacke@bth.se
gamescience.bth.se
www.acagamic.com
Connect at www.linkedin.com/in/nacke
Blekinge Institute of Technology
Box 214
SE-374 24 Karlshamn
Sweden