Markus Heberlein's talk from UX Talk Tokyo (uxtalktokyo.com) on 19 Feb 2014.

1. Qualitative Research Methods in
User Experience Design
UX Tokyo Talk
February 2014

2. What is Qualitative Research?
The data is not numeric
– Photos
– Videos
– Direct observation
– Comments on message boards
– Recordings of phone transactions
– Interview transcripts

3. What is Qualitative Research?
Although this data can be converted into
quantitative data, doing so is not always useful
or appropriate.
– Data of this type is usually very difficult to collect
consistently across a representative sample
– Meaningful valuable outcomes can be achieved
without using numbers
– This is especially good at providing you insights into
things you didn’t already know

4. What is Qualitative Research?
Analysis, interpretation, and presentation often are done
using very different techniques from quantitative
analysis.
Examples:
– personas - A "typical" user presented in a way that a
development team can focus on
– process maps - An understanding of the actual steps in a
process

5. What is Qualitative Research?
Some (mostly) qualitative research traditions from
the social sciences:
– Ethnography
– Biography
– Phenomenology
– Grounded Theory
– Case Study
– Ethno-methodology
Source:
Qualita've
Inquiry
and
Research
Design:
Choosing
Among
Five
Tradi'ons
by
John
Creswell

6. Why do Qualitative Research?
It is sometimes the best way to address research needs:
– Discovering issues that you don't know exist
– Sorting out messy information ( ie the critical knowledge only
exists in the heads of a privileged few)
– To complement quantitative research.
– Some projects do not allow for easy collection of quantitative
data (ie a B2B with a limited number of customers, hardware
development, or “sensitive” markets)

7. How to Start
Research should be designed carefully.
You need to consider:
– Budget
– Goals
– Available resources
– Schedule
– Etc...

8. How to Start
Good research starts with questions
Examples:
– How can we improve app X?
• What pain-points do current users have?
• What else would users like to do with the app?
– Which of these 5 innovation projects should we productize?
• What are the potential market, competitors, technical risks,
etc...?
• Which ones excite people?

9. How to Start
Know why your doing it
Often, you just want to bring back something for the
design team to make brain-storming more
productive.
But there can be other strategies

10. How much data to collect
How many visits to end users’ homes/offices is
enough?
It depends…
– Until you seen so much that you aren’t learning anymore
– Until you run out of time or budget
– Until you meet the objectives of the study

11. Some of the Tools
Semi-structured interviews
User observation
Photo & video “ethnography”

12. Semi-structured interviewing
Goldilocks interviewing style
Structured interviews tend to miss out:
– What you don’t know to ask
– Getting good detail without consuming too much time
Unstructured interviews tend to gather data that is
– Inconsistent
– Off-topic

13. Semi-structured interviewing
Usually requires more planning & refining
Create a research protocol
– An outline of what you want to talk about
– Format it so that you can be flexible while talking
– Pay attention to how you print it and use it in the field
Update it as you research
Like many other skills, it requires talent and practice.

14. Semi-structured interviewing
Recording
Where possible, audio-record
Get good at note-taking too
Working in pairs is also recommended, when possible

15. Semi-structured interviewing
Steering the conversation
Because it’s an interview, you are allowed to drive it in the
direction you want to.
But you also want to get them talking as openly, freely, &
much as possible.
Always be very aware of the time, but don’t make it too
obvious!!!

16. Semi-structured interviewing
Tricks:
Use pauses and interjections (相槌) effectively
Get them to tell stories
Get them to show you things

17. Observation
"What people say, what people do, and what they say
they do are entirely different things."
Margaret Mead

18. Observation
Depending on your topic, ability to observe and
video or photograph end users varies greatly
Strategies typically follow these constraints
You should always try to collect as much intelligible data
as you can, including what you store in your head

19. Observation
When you can’t use a camera….

20. Observation
Note what you can &
re-sketch before you
forget details

21. Observation
Frameworks
Typically useful for novice researchers or when
including engineers/designers in the research
process
– AEIOU (Actions, Environments, Interactions, Objects,
Users)
– POEMS (People, Objects, Environments, Message,
Services)
– Who, What, Where, Why, When

22. What to do with the data
Process it while it is fresh
If possible, everyone that participated should
sit down immediately following the
interviewing/observation, to discuss what
was seen.
Copious notes should be taken during this
session.

23. What to do with the data
If the goal of the research is deeper
understanding…
Deep analysis is needed.
Methods vary by researcher.
– Post-It sorts
– Video tagging
– Conversation analysis
Various, sometimes lengthy, reports result

24. What to do with the data
If the goal is quick fodder for brain-storming…
Make the data as clear and visual as possible
Try to bring the design team into the user’s
shoes