This document calls for rethinking traditional user research methods to better understand user behavior on the social web. It argues that (1) existing usability testing and other methods are outdated and don't capture how people use technology socially and contextually, (2) researchers don't know what they don't know so can't ask the right questions, and (3) new techniques are needed that combine field research and testing to understand relationships, networks and emergent behaviors online. The author advocates cultivating interdisciplinary "polymaths" and research designs focused on questions rather than templated studies.

1. Rethinking User Research for
the Social Web
Dana Chisnell @danachis
dana@usabilityworks.net #rethinking
1

2. Shelly
2

3. The state of the
art is a
generation old
Time for a new generation of
user research methods.
3

4. Usability testing isn’t telling us what we
need to know for designing for social
4

5. We don’t know how to find out about things
we don’t know about
5

6. People don’t live in the world doing one task
with one device out of context
6

7. http://www.pangaro.com/CUNY2008/index.html 7

8. http://www.pangaro.com/CUNY2008/index.html 8

9. Methods and measures were probably
always imperfect.
But they used to be enough.
No more.
9

10. Everything is
different now.
But not different.
All this has happened before.
10

11. The nature of
being online is
social
It always has been.
http://www.flickr.com/photos/holeymoon/ 11

12. Scale is a game
changer
Caution: large sample sizes
http://www.flickr.com/photos/daguerreotyped/ 12

13. Tasks aren’t
what you think
Activities = goals that emerge and
change
http://www.flickr.com/photos/jeremybrooks/ 13

14. Satisfaction is
correlated with
task completion
Instead: control, engagement
http://www.flickr.com/photos/litebriteneonstudio/
14

15. Users
continuously
design your UI
in real time
Workarounds = hacks
Etiquette and norms
http://www.flickr.com/photos/juleshabib/ 15

16. Social is about context.
And relationships.
16

17. Methods are not robust for understanding
context and relationships.
17

18. Cultivate polymaths
Decision sciences Communication
Anthropology sciences
Psychologies Sociolinguistics
Behavioral economics Organizational behavior
Psychobiology Sociology
Social neuroscience Social networks
18

19. The most successful user
research methods for SxD
combine field and testing
techniques
19

20. Methods
Reviewing online profiles, connections in groups in
interviews
Stories of how you met
Multi-user sessions with people who have strong and
weak ties
Video diaries with retrospective review
Experience sampling with SMS
20

21. Takes more time
recruiting, prep, session, analysis
21

22. Takes strong research design
rather than templated, commodity usability testing
22

23. Takes deep thinking
about questions to answer, and what
the evidence will be that you have answers
23

24. Takes study of cohesiveness
dynamics
collaboration, connections, networks, relationships
24

25. Where are you with planning?
What we asked answered a business question
25

26. Who do you ask when you don’t know?
Should have asked about context and relationships
26

27. Would have told us
Who is part of planning and deciding
Who is trusted
Why these people are important
Why they are trusted
How planning happens
27

28. To build a model that supports life
28

29. Rethinking user
research
We’re not getting the answers we need.
Experimenting is limited because we’re
pressured to go to market.
We’re looking for things we know about, using
old fashioned tools.
29

30. We’re missing things we don’t know about.
30

31. It’s time to
rethink user
research for
the new (old)
social web
31

32. Dana Chisnell
dana@usabilityworks.net
twitter: @danachis
www.usabilityworks.com
415.519.1148
32