Create great UX research plans using risk vs. effort mapping. Here's a practical framework with concrete examples showing how to apply it in any project.

1. Effort vs. risk mapping:
Creating foolproof research plans
Kea Zhang, Product/UX @Teston

3. Taking the hassle out of user testing

4. Interviews Fieldobservations
Moderated
usability testing
Unmoderatedusability testing
Diary studies
Surveys
A/B testing
Clickstream
analysis

5. Interviews Fieldobservations
Moderated
usability testing
Unmoderatedusability testing
Diary studies
Surveys
A/B testing
Clickstream
analysis

6. How the **** do I research this?!

7. Sorry, this isn’t a cheat sheet.

8. ...but effort vs. risk mapping can help

9. What we’ll talk about today
● What is effort?
● What is risk?
● What is effort vs. risk mapping?
● Using this to plan user research

10. What is effort?
The cost of building something.
This could be…
● Money
● Time
● Resources

11. What we’ll talk about today
● What is effort?
● What is risk?
● What is effort vs. risk mapping?
● Using this to plan user research

12. What is risk?
1. Level of uncertainty
2. Impact on the business

13. “There are things we know that we know.
There are known unknowns—that is to say, there
are things that we now know we don’t know.
But there are also unknown unknowns—there are
things we do not know we don’t know.”
Donald Rumsfeld, former US secretary of defense

14. Level of uncertainty
● What and how much do you know?
○ What are the knowns?
○ Known unknowns?
■ What do you need to know?
○ Are there unknown unknowns?

15. Level of uncertainty
● What and how much do you know?
○ What are the knowns?
○ Known unknowns?
■ What do you need to know?
○ Are there unknown unknowns?
● How new/innovative is it?
○ Are there existing designs and patterns to draw on?
○ Are you literally inventing the wheel?

16. Established design patterns

17. Established design patterns, in new ways

18. No patterns, unexplored territory

19. Level of uncertainty
● What and how much do you know?
○ What are the knowns?
○ Known unknowns?
○ Are there unknown unknowns?
● How new/innovative is it?
○ Are there existing designs and patterns to draw on?
○ Are you literally inventing the wheel?
The more you know, and the more you can draw on existing patterns,
the less uncertainty!

20. What is risk?
1. Level of uncertainty
2. Impact on the business

21. Impact on the business
What are the potential costs if this goes wrong?
What can you gain from doing this right?
For example…
● Money
● People
● Time
● Brand/company reputation

22. What assumptions are you making?
(Some assumptions are more costly than others…)

23. Some common assumptions...
● This product/feature has a competitive advantage
● Customers want to use/pay for this feature
● Users understand what this feature is and how to use it
● This feature will fit into our users habits, preferences and environments
● You have full organisational support for building this
● You’re solving the biggest/most important problem for your users

24. Checking your assumptions
What are the potential costs if you realise in 6 months:
● You don’t have a competitive advantage for this feature
● These features don’t matter to your customers
● Your users don’t understand what you’re offering
● You were wrong about users’ habits/preferences and environments
● You don’t have internal support for building this
● You’re solving the wrong problem

25. What is risk?
1. Level of uncertainty
2. Impact on the business
High level of uncertainty + high business impact = Greater risk

26. What we’ll talk about today
● What is effort?
● What is risk?
● What is effort vs. risk mapping?
● Using this to plan user research

27. What is effort vs. risk mapping?

28. What is effort vs. risk mapping?
Effort
Idea
Risk (uncertainty and impact on business)

29. What is effort vs. risk mapping?
Effort
Idea
Idea
Idea
Idea
Idea
Idea
Risk (uncertainty and impact on business)

30. What is effort vs. risk mapping?
Effort
Idea
Idea
Idea
Idea
Idea
Idea
Risk (uncertainty and impact on business)

31. High effort
Low risk
Low effort
High risk
High effort
High risk
Low effort
Low risk
What is effort vs. risk mapping?
Effort
Risk (uncertainty and impact on business)

32. What we’ll talk about today
● What is effort?
● What is risk?
● What is effort vs. risk mapping?
● Using this to plan user research

33. Low effort, low risk
Research before designing
● Desk research
During concepting/design/prototyping
● Usability testing
After shipping
● Clickstream analysis
● Monitor data/key metrics/conversions
Low effort
Low risk

34. High effort, high risk
Research before designing
● Field observations
● Interviews
● Surveys
During concepting/design/prototyping
● Usability testing (especially moderated)
○ Concept testing
○ Low-fidelity prototypes
Don’t think about shipping - continue until the project is low risk!
High effort
High risk

35. “Research should be designed to
address a problem — either to solve it
or to get closer to a solution.”
Emma Boulton

36. High risk → Low risk

37. High effort, low risk
Research before designing
● Desk research
During concepting/design/prototyping
● Usability testing
○ Ideally test multiple prototypes/designs
○ Test low-fidelity and high-fidelity designs
After shipping
● Clickstream analysis
● Monitor data/key metrics/conversions
High effort
Low risk

38. Low effort, high risk
Research before designing
● Interviews
● Surveys
During concepting/design/prototyping
● Usability testing
○ Focus especially on low-fidelity testing
After shipping
● A/B testing
● Clickstream analysis
● Monitor data/key metrics/conversions
Low effort
High risk

39. High effort
Low risk
Low effort
High risk
High effort
High risk
Low effort
Low risk
What is effort vs. risk mapping?
Effort
Risk (uncertainty and impact on business)

40. ...OK, I’ll offer you some general rules.

41. As a general rule
High risk?
● More research iterations
● Upfront research, before even starting design
● Focus on understanding…
○ The users - their habits, values, preferences
○ The problems
○ This means qualitative research
As you learn more, re-assess risk and adjust planning!
Low effort
High risk
High effort
High risk

42. As a general rule
Low effort, low risk?
● Less research
● Desk research often enough to start design
● Focus on evaluating/validating designs and tracking metrics after
release
○ This means quantitative research
Low effort
Low risk

43. Use this to structure conversations
around research planning!

44. Ask yourselves...
● How much effort will X take to build?
● How risky is it?
○ How much do we know vs. not know?
○ Are there existing UX patterns to draw on or not?
○ What assumptions/hypotheses do we have about X?
○ What is the business cost if we are wrong about our
assumptions/hypotheses?
This will help you align on where you are and what’s at stake before
deciding on methods.

45. Thanks for listening!
Kea Zhang, Product/UX @Teston
kea@teston.io