10. BAD RESEARCH GOAL GOOD RESEARCH GOAL
How can we make it easier
for employees to exchange
feedback?
How interested are employees
in exchanging feedback?
What would compel them to
exchange more feedback?
How can the feedback app
facilitate exchange of more
feedback?
:-( :-)
13. 1.Identify Target Users
1. Work at Pivotal
2. Are frequently involved in
team activities
3. Have used the feedback
app
4. Representative of gender
and ethnic diversity
14. 2.Find participants
Do what works.
- Contact existing users of the app
- In-app notice requesting participation in research
- Utilise a recruitment agency
- Contact your own network, friends and family
- Look around!
15. 3.Schedule Interviews
Using a tool like youcanbook.me or Doodle can help simplify
scheduling
- Allow a gap of 30 minutes between interviews
- No more than 5-6 interviews in a day
- Compensate the participants
- Tell the participants what to expect during the session
- Keep duration of interviews under an hour
24. What went wrong?
1. The table setup made it feel like a job interview
1. The tone of the questions felt accusatory
1. Going straight into the hard questions made Sherif uncomfortable
and defensive
31. What went wrong?
1. Leading question
“Do you think feedback is important, and necessary for a healthy
work culture?”
1. Close-ended, or yes/no answer questions
1. The dreaded “Do you think you would use feature X” question
45. Other tips
1. Look out for non verbal cues
1. Paraphrase what you have heard
“Let me rewind, and make sure I understood this correctly…”
3. Take a picture of the interview participant
3. Take comprehensive notes, transcribe word for word
3. Consider if you need participant’s consent
53. Further Reading
- Just Enough Research, by Erika Hall
- Observing the User Experience, by Mike Kuniavsky
- Remote Research, by Nate Bolt & Tony Tulathimutte
Hello everyone. I am Pranay, a Product Designer at Pivotal. You may know Pivotal from our cloud platform and data products, or the technology consultancy offering - Pivotal Labs.
Pivotal Labs has been around since the 1990s. We work with clients to help transform the way they build software by using agile, user centered design and extreme programming practices. We are located in the CHQ Building in Dublin, so feel free to drop by and say hi!
Today, Laura, Sherif and I will be sharing with you our recipe for a disastrous user interview, with the hope that you all will stay far away from it.
At Pivotal, we start projects with research and learning more about the users and their needs. The most common way we learn about users is through user interviews. Over the years, we have had our share of disastrous user interviews, and today we are going to share our learnings with you all.
In this presentation, we will walk you through the process of organizing a user interview based research, and demonstrate some disastrous mistakes to avoid during user interviews.
In an agile, user-centered product team, you should be reaching out to your users to get insights on the problems they have, how you can solve them using your product, and to validate if your product actually solves their problems or not. There are multiple ways to conduct this research. Let’s look at some of those Research Methods.
Qualitative Research is exploratory in nature, which you utilize to gain an understanding of behaviors and the underlying motivations and reasons that help explain the behavior.
Quantitative methods emphasize measurements and statistical analysis of data to gather insights.
For the purpose of this presentation, we will focus on Qualitative research, and specifically Interviews.
It’s important to drive in that we are not talking about usability testing. Usability testing is typically task driven as opposed to the more exploratory...
...open-ended nature of interviews.
And finally, we won’t be talking about just any open-ended interviews. We’ll be talking about...
… Disastrous open-ended interviews. But before we get to the disasters, we’d like to give you a quick run through of the activities you need to do to get started with a user interview based research project.
Alright. The first step is to set your research goals. The research goal helps frame the direction of your research, determine the right research method, and also helps you come up with questions that you’d be asking the participants of your research.
At Pivotal, we have a culture of exchanging feedback. We use a feedback app in order to do that. We took up a project to make it easier to exchange feedback, and exchange better beedback. You can frame a research goal for that project as - How can we make it easier for employees to exchange feedback?
But that’s a bad research goal. It is not granular enough and assumed that employees want to exchange feedback in the first place. Good research goals are specific.
Once you’ve got your research goals, the next step is to find participants to interview. But before you start contacting them, you need to know who your target users are.
Be aware that your source of participants will introduce biases, and different participant pools might represent slightly different needs and expectations.
Next step is to brainstorm questions that address each one of your research goals, and come up with an interview script.
A typical interview script starts with an introduction.
An easy introduction is important. Give the interviewee time to ease into the process. Introduce yourself and what you are looking to do in the session. Ask questions that are don’t require your interviewee to think much, but also give them cues about what the conversation is going to be about.
The body is the meat of your interview, where you ask questions that will lead you towards insights into your research goals.
And finally, the conclusion.
When concluding the interview, you would want to confirm the interviewee’s overall mood and have them reflect upon the experience. Ask them if they have additional thoughts. A good concluding question can be - “If you had a magic wand, and could change one thing about your experience, what would that be“.
It’s also a good idea to give them an opportunity to ask questions.
Typically, the script looks like a linear series of questions, broken down into themes. Nothing wrong with that. Just be cautious, not to let your interview feel scripted. The interview script is a bit of a misnomer. More than a script, it’s a checklist for your to refer to - to make sure you cover everything you want to during the interview.
Sometimes, we have used an alternate representation of the script, called a Topic Map. You jump from one topic to another, depending on where the conversation leads you.
Exploratory interviews are most useful when they happen like a free flowing conversation. Let your participant tell you a story.
Now we come to the fun part. Everything is scheduled. You have your first interview participant coming in. Nothing can go wrong now, right? Well, let’s see.
Let me invite Sherif here to play the role of an interviewee, our interview participant.
Laura will be the moderator, asking questions. And I will take notes. We will play out a few scenarios that demonstrate what *can* go wrong!
At this point, you want to evaluate if you have reached a good understanding of your user needs and have answers to the research goals you identified. You might want to organize another set of interviews, targeting a different persona, or continue to the next step where you can synthesize your findings and make use of them.
