Measuring emotional engagement is becoming an increasingly important aspect of interaction design. In order to truly measure the emotional impact that an experience is creating, UX researchers and designers need to have an understanding of methods that can accurately capture this information. Equally important to knowing what emotional responses are being created are ways to enhance or focus the emotional experience. A new branch of user experience research that includes physiological responses is on its way. This workshop will start a dialog on how these new tools can be applied to the UX field.

1. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 1
New Methods
for Measuring Emotional Engagement
Andrew Schall
@andrewschall

2. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall
Andrew Schall.
Vice President of UX, SPARK Experience
Co-Author, Eye Tracking in User Experience Design
President, UXPA of Washington DC

3. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall
A workshop is meant to:
React
to
Create
Refine
Ideas

4. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 4
HOW DO WE
EVALUATE EMOTION?
What methods exist to
understand how our users
are feeling?

5. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 5
You Can’t Accurately Ask About
Emotions
Current methods for
evaluating emotion are limited
to what a user selectively self
reports
to the study facilitator.

6. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 6
You Can’t Accurately Ask About
Emotions
Emotion is not primarily a
language-based experience;
conscious effort is required
to put experience into words.

7. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 7
BIO/NEURO METRICS
How can UX researchers
apply this technology to
study emotions?

8. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall
Disclaimer: I am not a
neuroscientist or human
physiologist.

9. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 9
EyeTracking.
Eye Tracking is critical for any
emotional analysis. Eye tracking
can tell us:
Time & Location
• Where they looked
• When they looked
Engagement Level
• How long they looked
• How frequently they looked

10. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 1 0
EyeTracking.

11. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 1 1
• HELPS YOU UNDERSTAND WHAT THE
USER IS PAYING ATTENTION TO
• EASY DATA COLLECTION
• SOFTWARE-GENERATED VISUALS
PROS
• TELLS YOU WHAT BUT NOT WHY
CONS
EyeTracking.

12. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 1 2
BrainActivity.
Electroencephalography is a
way to measure brain activity
with sensors on the head.
Brainwaves can be interpreted
to understand core emotions:
• Happiness/Sadness
• Excitement/Boredom
• Frustration/Relaxation

13. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 1 3
BrainActivity.

14. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 1 4
BrainActivity.
• BROAD SPECTRUM OF EMOTIONS
• ABILITY TO COLLECT VALENCE
• RELATIVELY FAST & EASY TO SETUP
PROS
• RAW DATA CAN BE NOISY
• TIME-CONSUMING & COMPLEX TO
ANALYZE
• DATA CAN BE MISINTERPRETED
CONS

15. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 1 5
SkinConductivity.
The sensor measures small
changes in user’s skin conductance
(GSR). These sweat gland changes
are directly related to the
Sympathetic Nervous System
activation, also known as the fight
or flight response.

16. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 1 6
SkinConductivity.
When a user becomes
frustrated, excited,
anxious, stressed, etc.
their skin conductance
will spike.

17. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 1 7
SkinConductivity.
• EASY TO SETUP
• MINIMALLY INTRUSIVE
• DATA ANALYSIS LESS COMPLEX
• FOCUS ON AROUSAL & STRESS
PROS
• DELAYED REACTION
• LIMITED RANGE OF EMOTIONS
• CAN NOT MEASURE VALIENCE
CONSFig. 2. Shimmer 3 GSR Unit (Source: Shimmer)
Researchers should be aware that the results from GSR do not correspond with a
response to a stimulus in real-time. GSR can produce response latencies between 3 to
6 seconds from the response to a stimulus8
. Therefore it is not recommended to use
GSR to identify the exact moment when a response was triggered. Instead,
researchers should analyze the response over a period of time, for example the
duration of a task or the presentation of a stimulus, and then compare the result to
other such units. For all biometric measurements, but especially with GSR, it is
critical to obtain a baseline measurement prior to the presentation of stimuli.
Participants will vary in terms of their typical level of sweat output and their
emotional state (e.g. feeling anxious) when they arrive at the test facility. By
establishing a baseline it provides a point of comparison between their state prior and

18. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 1 8
PupilResponse.
http://www.scientificamerican.com/article/eye-opener-why-do-pupils-dialate/
Pupilametry is used to measure the real-time
diameter of the changes in pupil size. This
can be used to detect emotions:
• Mental workload
• Frustration/Stress
• Arousal/Interest

19. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 1 9
PupilResponse.
Research shows that the worst
time to interrupt anyone is when
they have the highest memory
load. Pupillometry studies have
shown that interruptions during
peak loads cause the biggest
disruption.
Shamsi T. Iqbal, Xianjun Sam Zheng, and Brian P. Bailey. 2004. Task-evoked pupillary response to mental
workload in human-computer interaction. In CHI '04 Extended Abstracts on Human Factors in Computing Systems

20. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 2 0
PupilResponse.
• EYE TRACKER ALREADY
COLLECTS IT
• STRAIGHTFORWARD ANALYSIS
• FOCUS ON AROUSAL & STRESS
PROS
• HIGHLY SUBJECT TO
LIGHTING VARIABILITY
• CAN NOT DETECT VALENCE
CONS

21. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 2 1
FacialAnalysis.
Facial expression recognition uses a simple
webcam and software to identify facial
expressions people display.
It can detect a wide range of emotions:
• Happiness
• Sadness
• Angry
• Frustration
• Confusion

22. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 2 2
FacialAnalysis.
The software looks at facial action units that are then combined to
identify specific emotions.

23. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 2 3
FacialAnalysis.
• AUTOMATIC CODING OF FACIAL
EXPRESSIONS
• CAN DETECT VALENCE
• WORKS WITH ANY WEBCAM
PROS
• INCONSISTENT RESULTS
• MAY REQUIRE LARGER
SAMPLE SIZE TO ACCOUNT
FOR VARIANCES
CONS

24. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 2 4
DataAnalysis.
iMotions Attention Tool Noldus Research Suite

25. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 2 5
TakeAways.
Major opportunities to use bio/neurometrics to understand
users’ emotions in the context of user experience.
Tools are rapidly improving in terms of easy of data collection
and analysis…but still a long way to go.
Be aware of the current limitations and challenges of using
this technology.
01 /
02 /
03 /

26. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 2 6
YourVoice.
What are the best applications for biometric
measurements in UX research?
What are the practical considerations/issues
with using these tools for UX research?
How do we design an effective study to
measure emotion for UX evaluation?
Is this the future of UX research?
01 /
02 /
03 /
04 /

27. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 2 7
GROUP ACTIVITY
What are the best applications
for biometric measurements in
UX research?

28. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 2 8
GROUP ACTIVITY
What are the practical
considerations/issues with using
these tools for UX research?

29. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 2 9
GROUP ACTIVITY
How do we design an effective
study to measure emotion for UX
evaluation?

30. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 3 0
GROUP ACTIVITY
Is this the future of UX
research?

31. UXPA 2014 New Methods for Measuring Emotion | June 25, 2014 @andrewschall 3 1
OPEN FORUM
What are your ideas, questions,
concerns about this kind of
research?