Top profile Call Girls In Mau [ 7014168258 ] Call Me For Genuine Models We ar...
Moderating to the Max: Refining Your Interviewing and Moderating Skills
1. Susan Mercer – Senior Experience Researcher
@susanamercer
May 29, 2013
UXPA Boston 2013 Conference
Moderating to the Max
REFINING YOUR MODERATING &
INTERVIEWING SKILLS
2. Hello, I’m Susan Mercer
BA and MSc in Geophysics
19 years in software and web UI and UX design
Developer
Designer
Web Producer
Product Manager
Researcher
MS Human Factors, Bentley University
Twitter: @susanAmercer
2
4. Moderating and Interviewing Skills
On paper, they seem straightforward
Write a screener
Recruit
Write a script
Conduct the study
Once I ran a couple easy ones, I thought I had it down.
4
Interviews
Formative
Usability
5. But Then…
I got the participants who…
And I learned an important lesson:
Interviews and usability tests are
more than just a series of questions.
5
6. My Epiphany
Don’t play the notes on the page…
…play the music
This applies to moderating and
interviewing too.
Don’t just read the questions from the page…
…have a conversation
6
7. What Is Our Goal?
To elicit honest thoughts from participants in an unbiased manner.
Sometimes it’s hard to be fully honest.
7
8. The Rules of Politeness
We’re often polite rather than honest to strangers
Social Acceptance > Honesty
8
9. How Do We Get Beyond Politeness?
Build their trust
9
11. Participant Comfort
Two schools of usability testing moderating techniques
Moderator keeps silent and says “Keep talking” 1
Awkward feeling for participant
Reminds them that they are in a “study”
Moderator creates conversation where participant is primary
speaker 2
Still accomplishes goals of gathering information
Meets the participant’s expectations of a socially acceptable conversation
Moderator can still be neutral and minimize bias
11
Sources: 1 Ericsson and Simon, 1980, 2 Boren and Ramey, 2000.
12. Conversations
• We carry on conversations all of the time
• We convey a lot of subtle meaning in our conversations
• We have rules for conversation
• We know what is acceptable in which situations
12
13. What Do We Do in Usability tests and interviews?
We ask participants to:
Come into a usability lab / meet us in a strange location
Talk to a stranger about important topics
Use a product or website and make mistakes
While being observed
Without knowing the right social norms on how to behave
13
14. To Do List
• Elicit honest feedback
• In an unbiased manner
• Instill trust in us
• Make the participant comfortable
• …and get them to answer all of your questions
14
16. Structure of an Interview/Usability Session
16
Trust
Comfort
Interview
Build Rapport
Neutral
Observation
Define Social
Interaction Rules
Be Transparent
Comfortable
Conversation
Be Accepting
Thanks
Manage the Flow
Greeting Introduction
Interview /
Evaluation
Wrap-Up
17. 1. Greeting – Building Rapport
Be a Friendly Person
Smile
Use their name
Be a good listener
Make the other person feel important – and do so sincerely
A person's name is to that person the
sweetest and most important sound in
any language.
- Dale Carnegie
18. 1. Greeting – Building Rapport
Small Talk – Find Common Ground
Safe topics: travel to office, traffic, weather, local sports
Avoid asking direct questions
Listen and look for shared experiences
19. 1. Greeting – Building Rapport
Be Empathetic
Apologize if they had trouble finding the office
“Oh, it’s raining there? It is here too. I hate rainy days.”
Show you understand their point of view
“I can understand that…”
“I can see that…”
“That does sound very frustrating…”
20. 1. Greeting – Building Rapport
Inject Some Humor
Joke about yourself
Joke about the situation
Don’t joke about them
Rapport is about showing them that you are human too
21. 2. Introduction – Be Transparent
Confidentiality
How are you recording what is said?
Who is listening?
What will you do with the information you collect?
Consent Form
Put it all in writing – using their language
22. 2. Introduction – Define Social Interaction Rules
Describe the Session
List the activities
Describe the roles
Social Niceties Do Not Apply
You’re not emotionally involved in the design/project
There are no right or wrong answers
Your job is to get honest opinions
23. 3. Interview/Evaluation – Be Accepting
Watch Your Reactions
Don’t show surprise
May make them think that they are giving a wrong answer
Don’t overly agree
May make them think that they are giving the right answer
Don’t be negative
Watch your tone – stay neutral and accepting
Try not to laugh
24. 3. Interview/Evaluation – Be Accepting
Be yourself
No one is perfectly neutral
Recover gracefully and move on
“Perfect” – “That’s the level of detailed feedback we’re looking for.”
“Interesting!” – “I haven’t heard that perspective yet, tell me more.”
(something surprising) – “I can understand that.”
Interject some Rapport-building comments when needed
Quiet or uncomfortable participants
“I hate it when that happens.”, “I can imagine that was challenging”, etc.
Again, showing that you are human like them
25. 3. Interview/Evaluation – Comfortable Conversation
What is a Comfortable Conversation?
Conversational cues and turn-taking are expected
Acknowledgement tokens – “Uh huh”, etc.
Encourage the continuation of the other speaker’s talk
Usually implies that the other speaker’s prior talk is incomplete
Source: 1 Drummond and Hopper, 1993.
26. 3. Interview/Evaluation – Comfortable Conversation
Some acknowledgement tokens indicate other things
“Yeah” – half the time indicates changing turns in speaking 1
“Yeah. I heard that the other day and…”
“Oh!” - may indicate noticing something, then transitioning to another
topic
“Oh! That reminded me…”
“Okay” – may indicate that the listener wants the speaker to stop
speaking1
Source: 1 Drummond and Hopper, 1993.
27. 3. Interview/Evaluation – Neutral Observation
Some may introduce bias
“Oh!”, “Interesting” – indicating unexpected answer
“Yes”, “Perfect”, “Great” – indicating agreement
“Hmmm.”, “Really?” – indicating disagreement
Notice that tone is key
Neutral is best
“Mhmm”, “Uh huh”, “Continue”, “Tell me more”, “OK”
“Mhmm” or “Uh huh” vs. silence interviewees saying 31% more
phrases. 1
• Body Language
• Head nodding while participant is speaking interviewees speak 50%
longer. 2
Source: [1] Matarazzo et. al., 1964, [2] Matarazzo et al., 1963
29. 3. Interview/Evaluation – Neutral Observation
Really Listen
Pay attention – stay in the moment
Look at the participant
Take notes if you can
Be quiet - give them time to say what they need to
30. 3. Interview/Evaluation – Neutral Observation
Be Quiet!
Most agreements happen immediately. Most people delay before
disagreeing.1
If you don’t respond to their answer, it encourages them to talk more
People often delay speaking before disagreeing – give them time
Some people are uncomfortable with silence, so they will keep talking
“People speak in paragraphs.” (Steve Portugal)
The best way to stay neutral
Source: 1 Goodwin and Heritage, 1990.
31. 3. Interview/Evaluation – Manage the Flow
• Wait for topics to come up organically
• If the participant mentions something that you have later in your question
list, ask it now.
• Transitions
• Don’t just switch topics – provide context.
• Summarize last topic, then introduce new one.
• “That’s great about XYZ, but now I’d like to talk about ABC”. Question.
• “You mentioned XYZ, tell me a bit more about how you blah blah..”
32. Structure of an Interview/Usability Session
32
Trust
Comfort
Interview
Build Rapport
Neutral
Observation
Define Social
Interaction Rules
Be Transparent
Comfortable
Conversation
Be Accepting
Thanks
Manage the Flow
Greeting Introduction
Interview /
Evaluation
Wrap-Up
35. Keep Improving
1. Identify your improvement areas
Watch your videos
Have colleagues give you feedback
35
36. Keep Improving
2. Learn from others
Watch others moderate
Be a participant
Listen to talk radio interviews
Terri Gross, Fresh Air, Ira Glass, This American Life
36
37. Keep Improving
3. Practice skills in everyday life
People watch (observe) in public
Be quiet and listen in everyday conversations
37
40. Coming Soon!
New book on Moderating
• Donna Tedesco, Fiona Tranquada
• Book coming this Fall
• Follow @ModSurvivalUX
See them today:
• “Expecting the Unexpected: Preparing for Successful User Research
Sessions”
• 3:00 PM, Back Bay C
40
43. References
Boren, T. and Ramey, J. (2000) Thinking aloud: reconciling theory and practice. IEEE Transactions on Professional
Communication, 43 (3), 261-278.
Carnegie, D. (1936) How to win friends and influence people. Simon & Shuster.
Dumas, J. and Loring, B. (2008) Moderating usability tests, Morgan Kaufman.
Dumas, J. and Redish, J. (1999) A practical guide to usability testing, Intellect Ltd.
Ericsson, K. and Simon, H. (1980) Verbal reports as data. Psychological review. 87 (3), 215-251.
Goodwin, C., & Heritage, J. (1990). “Conversation analysis.” Annual review of anthropology 19 (1990): 283-307.
Matarazzo, J.D.., Saslow, G., Wiens, A. N., Weitman, M. & Allen, B. V. (1963). Interviewer head nodding and interviewee
speech durations. Psychotherapy, 1, 54-63.
Matarazzo, J.D.., Wiens, A. N., Saslow, G., Allen, B. V., & Weitman, M. (1964). Interviewer Mm-Hmm and interviewee
speech durations. Psychotherapy: Theory, Research & Practice, 1(3), 109.
Tannen, D. (2005). Conversational Style: Analyzing Talk among Friends. Oxford University Press, USA.
http://www.uxbooth.com/articles/5-useful-lies-to-tell-user-research-participants/
http://www.skillsyouneed.com/ips/rapport.html
43
Here to talk about moderating and interviewingInterviews and Usability tests are often done in different parts of the project lifecycle, but they are quite similar in the skills requiredFormative done during the design process, to get qualitative feedback used to iterate on the designSummative done at the end of development. More structured, quantitative, to measure specific aspects of the softwareInterviews and Formative tests are similar in structure and skillsets needed to effectively moderate them. That’s what I’ll be focusing on today.
There are lots of soft skills involved.
“Would you like to try one of my chocolate chip oyster and herring brownies? They’re really good!”
Rules of Politeness govern social conversations – and are stronger with people we do not know wellDon’t Impose“Would you like something to drink?”“Yes, that would be nice.”VS“Yes, I’ll have an Orange Juice mixed with Raspberry Lime seltzer.”Give Options“Would you like something to drink?”“Water, tea, or whatever you have – don’t go to any trouble.”Be Friendly“What do you think of this website?”“Well, it’s not my favorite…”VS.“This is crap.”
A warm smile is the universal language of kindness – William Arthur Ward