Based on 60,000+ hours of research from Nate Bolt, founder of Ethnio and former UX Research Manager at Facebook.

1. Best Practices From 10 Years of
Remote UX Research

2. thx

3. UX Research
vs.
Remote UX Research
vs.
User Research

4. Watching people use
interfaces in order to
design better interfaces

5. 60 YEARS AGO

6. LET’S
RECORDHUMANSUSING
INTERFACES
30 YEARS AGO

7. Vs.
10 YEARS AGO

8. 1%
Humans
observing
humans
99%
Data

9. • UX Research (Facebook, Google)
• Experience Research (AirBnB)
• Insights (Netflix)
• Design Research (Uber)
• User Research (So Many People)
INTERCHANGEABLE TERMS

11. Analytics
Data Science
Quantitative
Research
User Research
UX Research
Usability
User Research
1:1 Observation
Lean Research

12. TEXT
Geography of remote UX Research

14. Btw I’m Nate Bolt

17. PLEASE BUY
THIS BOOK

19. ethn.io

20. How To
Plan, Recruit, and Conduct
Remote UX Research

22. 1. Ask the right questions
AKA Conversations with Stakeholders

23. Figuring out the right
question(s) is tricky

24. Quantitative research lets us
test our assumptions, and
qualitative research lets us find
out what we don't know.
“
”

25. What do our customers/
users/humans want?
“
”

26. • Remoteresear.ch site
• Mental Models by Indi Young
• Just Enough Research by Erika Hall
• Cognition in The Wild by Ed Hutchins
• Complete Beginner's Guide to Design Research
• Observing the User Experience: A Practitioner's
Guide to User Research
A Deeper Read

27. 2. Try to Incorporate Data
•Surveys
•Experiments (A/B, etc)
•System data
•Purchase Behavior

28. 3. Identify
Testing
Goals

29. What do our customers/
users/humans want?
“
”

30. (and write
them down in
a G&O and
then in a
Moderator
Script)

31. remoteresear.ch/category/samples

32. 3. Recruit real humans
ethn.io

33. The Recruiting
Spectrum
Grab Someone
in Your Office
Live Customer
Intercept
Recruiting
Agency

34. 4. Combine Tools & Methods

35. Observe Behavior

36. 5. Collaborate don’t
report

37. And how does this fit into the
design process?

38. Understand
Prototype
Build

39. Understand
• review existing research
• focus groups
• interviews
• ethnography
• surveys
• card sorts
• data analysis
• diary studies

40. Prototype
• participatory design
• collaborative design
• brainstorming techniques
• heuristic evaluation
• paper prototype testing
• static mock testing

41. Build
• usability testing
• heuristic evaluation
• diary study
• A/B test
• field testing

42. 1. Identify Testing Goals
2. Incorporate Data
3. Recruit Real Humans
4. Combine tools & methods
5. Collaborate Don’t Report
How To
Plan, Recruit, and Conduct
Remote UX Research

43. Pitfalls
And Now Some Common

44. Decision Theory & Adaptive Systems Group

46. • Hypothetical Questions
• A False Premise
• Opinions are worthless
Some Common
Pitfalls
of Remote UX Research

47. • Behavior
• Context
• Behavior
Best Practices From 10 Years of
Remote UX Research

48. • Bandwidth (games)
• Security (prototypes)
• Translation (time)
• Rapport (lacking)
Even More
Pitfalls

49. How&When to Run Rapid
Remote UX Research

51. 46 Million Americans
That’s 22 Million Households
15% of the country

53. Now what percentage of
those 46 million were
employed within a year of
receiving food assistance?

54. > 80%

55. So these are working
people. People with
families. Teachers, etc.
How do they apply?

56. 90 Minutes / 53 Days
up to

57. Alan Williams
Design Lead, SNAP
Caroline Hays
Health Focus Area Intern
Jacob Soloman
Policy analyst and project manager

58. SNAP(Supplemental Nutrition Assistance Program)
Let’s Try the Interface for a
Application

62. Just Stop

66. 30 phone interviews
10 minutes each
58 unique experiences in California

67. And 15 Follow-Up Phone
Interviews

68. Quant & Qual

73. 11 Minutes / 53 Days (?)

74. • 30 Remote Interviews
• Real context
• 15 Follow-Ups
• Little to No Report
• Input straight to designer & engineer
Best Practices From 10 Years of
Remote UX Research

75. • Easy, cheap, & fast phone interviews
• Quick “Guerilla” Human Observation
• No statistical significance
• Behavior repeats over small samples
• Combine with other inputs
Best Practices From 10 Years of
Remote UX Research

76. Top Tools for
Remote UX Research

77. “At least a little
behavioral”
- Marv
Surveys.
Web Analytics.

78. Five Categories
•Mobile (spans other categories)
•Moderated (screen-sharing + audio)
•Self-moderated (usertesting.com)
•Automated (loop11, usabilla)
•Fancy Analytics on Steroids (crazyegg)

79. remoteresear.ch/tools

80. Conceptual
For A Long Time
Self-Moderated
Moderated
Web Analytics
Automated Static
Automated Live

81. Tasks On Concepts
For A Long Time
People Talk To A Machine
Talk To People
Fancy-Ass Analytics
Task Bar On Images
Task Bar On A Live Site

82. I.A. & Navigation
For Ad Agencies
Ammunition
Insight
Just Better Analytics
Still More Ammunition
More Ammunition

83. Conceptual
For A Long Time
Self-Moderated
Moderated
Web Analytics
Automated Static
Automated Live
Deep Behavioral
Ux Insight

84. Opinions
Conceptual
For A Long Time
Self-Moderated
Moderated
Web Analytics
Automated Static
Automated Live

85. Ad Agencies
Love It
Conceptual
For A Long Time
Self-Moderated
Moderated
Web Analytics
Automated Static
Automated Live

86. USABILLA
Clicks & Questions
Heatmaps
You do your own recruiting
$49 - $199

90. ethn.io

96. If NASA can do
THIS from 153
million miles
away, we can
meet our users
where they are,
even if we’re a
couple thousand
miles away.

97. Thanks for attending
Go run some awesome remote UX research

98. fin.

99. • Remoteresear.ch site
• Mental Models by Indi Young
• Just Enough Research by Erika Hall
• Cognition in The Wild by Ed Hutchins
• Complete Beginner's Guide to Design Research
• Observing the User Experience: A Practitioner's
Guide to User Research
A Deeper Read

100. nate@ethn.io
@ethnio
Nate Bolt