Focus groups can help marketing teams figure out what consumers want so they can build the right products for them. But this methodology has been co-opted by user experience teams, who incorporate traditional UX exercises into focus group sessions. The argument against focus groups involves the inability to predict future behavior, the great divide between what users say and what they do. The argument for them is that with preparation and a skilled moderator, they provide actionable results. Can we UX researchers rely on focus groups to drive design and strategy? Or are they the wrong tool for the job?
Bob Thomas believes that focus groups belong in the UX toolkit. They can provide high-level results that enable companies to make strategic decisions around product direction, early in the development cycle. Their success depends on preparation and good moderator skills.
Bob Thomas is Manager of User Experience at Liberty Mutual, where he has worked to expand usability best practices and build a UX team. He has presented at local and national UXPA conferences. He holds an MS in Human Factors in Information Design from Bentley University, and an MBA from Suffolk University. He is on the Board of Directors of New Hampshire UXPA.
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Five Arguments Against Focus Groups - And How To Overcome Them
1. 2013
Five Arguments Against
Focus Groups — And How
to Overcome Them
Bob Thomas
Manager of User Experience, Liberty Mutual
robertl.thomas@libertymutual.com
@bobthomas
2. 2013
Five Arguments Against Focus
Groups
1. You spend your time listening to users
instead of observing their behavior
2. Focus groups are ruined by dominant
participants and group think
3. Focus groups rely on users’ abilities to
predict their future behavior
3. 2013
Five Arguments Against Focus
Groups
4. It’s wrong to allow users to design
products in focus groups
5. Focus groups are selected as a
methodology, because everyone
understands them
5. 2013
Argument 1: You spend your time listening
to users instead of observing their behavior
๏ Change a flaw to a strength: Make focus
group participants do things
6. 2013
๏ Then build a discussion around activities
your participants just completed
7. 2013
Argument 2: Focus groups are ruined by
dominant participants and group think
๏ Control the room: set the ground rules early
11. 2013
Argument 3: Focus groups rely on users’
abilities to predict their future behavior
๏ Change a flaw to a strength: Recruit
participants based on past experience
๏ Build discussions around task-based
scenarios
Primary Recruit: Have filed an auto
insurance claim within the last 12 months
12. 2013
๏ Ultimately you want to base your
strategic direction on what users say they
will do, based on that past experience
13. 2013
Argument 4: It’s wrong to allow users in
focus groups to design products
๏ Change the focus: Show them designs and
get their reactions
54% negative words, including
“busy” and “overwhelming”
14. 2013
๏ Focus groups let you showcase ideas
that you can then usability test
Session 1
Claim via
phone call
Claim via
mobile app
Claim via
tablet app
Claim via
computer
Research
claims online
Claim via
live chat
Someone Injured 6 1 0 0 0 0
Car Towed 7 0 0 0 0 0
Car Drivable 6 0 0 0 1 0
First Accident 6 0 0 0 1 0
Second Accident 5 0 0 1 1 0
Totals 30 1 0 1 3 0
15. 2013
๏ Focus groups don’t help you make design
decisions. They help you make strategic ones.
16. 2013
Argument 5: They only become the methodology
of choice because everyone understands them
๏ Make them work for you: OK, so
management understands focus groups.
Why not use them to sell UX?
17. 2013
๏ Management loves focus groups. It’s a
mental model that makes sense to them.
Source: www.CartoonStock.com
18. 2013
๏ Focus groups can end the opinion wars
Source: Photo courtesy of Schlesinger Associates’ New Jersey facility
19. 2013
Conclusion
1. You spend your time listening to users
instead of observing their behavior
2. Focus groups are ruined by dominant
participants and group think
3. Focus groups rely on users’ abilities to
predict their future behavior
4. It’s wrong to allow users to design
products in focus groups
5. Focus groups only become the
methodology of choice because
everyone understands them
1. Make users do things
2. Prepare. Set ground rules. Facilitate.
3. Recruit based on past experience
and then create scenarios
4. Use them to confirm and deny ideas
5. Rely on them to end the opinion
wars
5 Arguments Against Focus Groups 5 Ways to Overcome Them
20. 2013
Let’s keep focus groups in the
UX toolkit. Thank you!
2013
Bob Thomas
Manager of User Experience
Liberty Mutual
E-mail:
robertl.thomas@libertymutual.com
Twitter:
@bobthomas
Source: Original photo courtesy of the Abraham Lincoln Art Gallery
Notas del editor
There seem to be two camps when it comes to focus groups: Those who think our research should be based only on what you can observe, and those who think you can also base it on what people say So let’s see how we can respond to these 5 arguments against focus groups
If the argument against focus groups is that you spend your time listening to users instead of observing their behavior – then make users do thingsFor example, give people “funny money” and have them Vote with their dollars on features they’d like to see
You can have users do tasks silently by themselves: complete short surveys, five seconds tests of designs, or vote on which of several designs they like bestAnd then you can build a discussion around that activity that they just completed
If the argument against focus groups is they’re damaged goods b/c 1 or 2 people dominate and everyone agrees with themThen make sure you control the roomI always spend time at the beginning of the session, setting expectations and the ground rules, including rules of the focus group.
It takes experience to facilitate discussions. You need to shut off loud and persistent participants, and reach out to others for feedback, including quiet participants. The best discussions are those where participants in a focus group dynamically interact with each other.
Summarize what participants say so you can recap key discussion points: Have a facilitator from your team in the room to take notes on a white pad, making sure that all participants’ have their words recorded.
And I can’t emphasize this enough, Prepare effectively: write a good moderator’s guide and test plan that you’ve discussed and vetted out with your business partners. And do practice runs of focus group with volunteers from your company, where people can role play
If the argument against focus groups is you can’t base decisions on what people say they’ll do, because people are more concerned with projecting the right self-imageTHEN Recruit participants based on their past experience and design scenarios around that experience.
… YOU can build your discussion points around task-based scenarios, just as you would in a usability test. For example, we’ll show participants who’ve had to file an insurance claim a simple diorama of cars getting into a fairly common accident. And then we’ll discuss what they’d do.
If the argument against focus groups is we shouldn’t allow users to design our products THEN don’t let them. Show them designs and get their reactions. For example, the home page for our mobile site was busy and overwhelming. We knew it but needed to prove it.
We should use focus groups to confirm or deny ideas, and then usability test those. For example, after a car accident, no one wants to file a car insurance claim on a computer. People want to call. We needed to confirm this and then we could usability test new solutions.
Focus groups help you steer the vision. Ultimately, they help you make strategic decisions.We’ve built out new home page designs for our mobile site, based on focus group results. And then we’ve usability tested those new designs.
If the argument against is they become the methodology of choice only because everyone understands them. You can still make them work for you in the right situationsHave you ever listened to management try to explain to us what we do? They say things like, You do QA, right? You make things pretty, right?
But focus groups represent a mental model that makes sense to management.You gather everyone in one place, and management comes to those sessionsThey can see how teams collaborateIt’s a great way to sell UX
As with usability testing, focus groups can end the opinion wars. You can run 5 focus groups in 2 days with 40-60 participants, quickly and at less cost than running the same number of usability tests .So let’s review the 5 ways to make focus groups work for you.
So let’s keep focus groups in the UX toolkit. Thank you, Washington DC and my fellow UX professionals. Again, I’m Bob Thomas, Manager of User Experience at Liberty Mutual. You can e-mail me at robertl.thomas@libertymutual.com or contact me on Twitter @bobthomas