1. COVERT AND OVERT
measures
of
engagement
within
an
educaNonal
mulNmedia
environment.
This
research
was
supported
by
Office
of
Naval
Research
under
Grant
N00014-­‐10-­‐1-­‐0143
awarded
to
Dr.
Robert
Atkinson
Robert
M.
Christopherson,
Javier
Gonzalez-­‐Sanchez,
Mustafa
Baydogan,
Maria
Elena
Chavez-­‐Echeagaray,
David-­‐Gibson
Robert
Atkinson
lsrl.lab.asu.edu

2. Games
can
change
the
way
we
learn
Empirical
research
can
change
the
way
we
game

3. Learning
Gaming
surprise
happiness
flow/engagement
anger
delight
boredom
frustraNon
confusion
curiosity
anxiety
fear

4. Goals
EDUCATION:
Engagement
=
Learning
GAMES:
Engagement
=
Fun

5. Overt
-­‐
observable
Covert
-­‐
hidden

6. Overt
and
Covert
Facial expressions
Cognition
Verbalization
Motivation
Behavior
Flow
Engagement
Attitude
Performance
Emotion
Physical interactions
Posture

7. What
is
engagement?
GAMING
“concerned
with
all
the
qualiNes
of
an
experience
that
really
pull
people
in
–
whether
this
is
a
sense
of
immersion
that
one
feels
when
reading
a
good
book,
or
a
challenge
one
feels
when
playing
a
good
game,
or
the
fascinaNng
unfolding
of
a
radio
drama”
Benyon
and
colleagues
(2005)

8. What
is
engagement?
LEARNING
“the
nexus
of
intrinsic
knowledge
and
interest
and
external
sNmuli
that
promote
the
iniNal
interest
in,
and
use
of
a
computer-­‐based
learning
environment”
(Jones,
1998)

9. Measuring
Engagement
OBJECTIVE
ObservaNonal
Physiological
CogniNve
Analysis
data
walkthrough
Think-­‐aloud
QUALITATIVE
HeurisNc
EvaluaNon
QUANTITATIVE
Surveys
&
Interviews
&
QuesNonnaires
Focus
Groups
SUBJECTIVE

10. Use
of
Physiological
Data
1. Decide
what
you
want
to
measure
2. Choose
the
appropriate
sensors
3. Control
your
task
and
environment
4. Process
the
data
according
to
which
sensors
were
chosen
5. Make
inferences,
evaluate
and
revise

11. 1.
Decide
what
you
want
to
measure
• engagement
• arousal
• mental
effort
• ajenNon
• excitement
• boredom
• meditaNon
• frustraNon

12. 2.
Choose
the
appropriate
sensors
PHYSICAL
PSYCHOLOGICAL
skin
conducNvity
(GSR)
arousal
pressure
on
controller
frustraNon,
arousal
posiNve
vs
negaNve
emoNons,
pupil
dilaNon
mental
effort
engagement,
mental
effort,
brain
waves
(EEG)
frustraNon,
boredom
gaze
locaNon
ajenNon

13. 3.
Control
your
task
and
environment
Con
• 20
Users
play
Guitar
HeroTM
• easy
and
hard
song
• 15
mins
of
pracNce
• Skin/Eye/Head/
Guitar
sensors

14. Real
Time
Monitoring

15. 4.
Process
the
data
according
to
which
sensors
were
chosen

16. 5.
Make
inferences
and
iterate
on
game/
instrucNonal
design
VisualizaNon
InterpretaNon
Data
StaNsNcs
mining

17. High
Scoring
Performance
in
Guitar
HeroTM
–––––
Raw
Engagement
–––––
Median
Engagement
–––––
Normalized
Performance

18. Low
Scoring
Performance
in
Guitar
HeroTM
–––––
Raw
Engagement
–––––
Median
Engagement
–––––
Normalized
Performance

19. Engagement

20. Boredom

21. MeditaNon

22. FrustraNon

23. Long
Term
Excitement

24. Why
measure
engagement?
• Dynamic
Difficulty
Adjustment
• Expand
Demographics
• Longer
Time
on
Task

25. Dynamic
Difficulty
Adjustment
CHALLENGE
Anxiety
Flow
Zone
Boredom
ABILITY

26. Expand
Demographics
CHALLENGE
Expert
Flow
Zone
Novice
ABILITY

27. Time
on
Task
FLOW
TIME

28. ROI
in
Games
• CompeNNve
edge
• Broader
appeal
• Micro
and
macro
evaluaNon
• PersonalizaNon
• Improve
gameplay

29. ROI
in
EducaNon
• Increase
Performance
• RetenNon
• Time
on
task
• Antude
toward
learning

30. Ongoing
Work
• SeducNve
Details
(InstrucNonal
Design)
• Videogames
and
Engagement
(Guitar
Hero)
• EmoNons
and
Working
Memory
Capacity
(puzzles)
• 3D
InstrucNonal
training
(US
Navy,
Save
Science)
• AffecNve
Meta
Tutor

31. QuesNons?
lsrl.lab.asu.edu