This document discusses how to quantify user experience research findings to persuade stakeholders. It provides two case studies: 1) Validating a new website navigation structure using tree testing which found the new structure was 40% better. 2) Measuring expected success of a new concept for Philips' website by testing tasks, satisfaction and goals on prototypes which provided convincing results to support investment. The key takeaways are to use quantitative analyses in design to facilitate objective communication with stakeholders and identify issues before development.

1. Stakeholder Persuasion
How to quantify your gut feeling!
Jacco Nieuwland & Martijn van Loon
March 2013

2. Why change something
that already works?
http://upload.wikimedia.org/wikipedia/commons/a/ac/United_States_Federal_Reserve_Board%2C_1917.jpg

3. Why this
colour?

4. Are we really
sure this design
is better?

5. researcher
user research reports
user tests
requirements requirements
user Product business
design
presentations
designer

6. user
flow
efficiency
mental
researcher
model
ROI
business talk;
# and $ KPI
UX talk /
designer talk
conversion
looks better
business
user
experience
intuitive
designer

7. researcher reports
?!?
UX talk /
designer talk
business
presentations
designer

8. UX & business - different languages
Need numbers to persuade
stakeholders
insights
Collect data during design process

9. Martijn
van Loon
@martijn_vanloon
Researcher
Jacco
Nieuwland
@jnieuwland
Designer

10. We are User Intelligence
an international User Experience
consultancy, specialized in
Research,
Design &
Strategy

11. researcher reports
Business talk;
# and $
UX talk /
designer talk
business
presentations
designer

12. Why change something
that already works?

13. Existing website: Web Traffic Map

15. understand conceive design implement optimize
web web
analytics analytics
Collect (new) data

16. http://upload.wikimedia.org/wikipedia/commons/thumb/4/45/Work_tools.jpg/1280px-Work_tools.jpg

17. Two cases

18. Case 1
Validate new navigation structure

19. The problem
I can’t find
the content
user product

20. Is the new
structure
better?
Can customers find
products more easily?

21. http://upload.wikimedia.org/wikipedia/commons/thumb/4/45/Work_tools.jpg/1280px-Work_tools.jpg

22. Current structure New structure

23. Tool: Treejack
Task: e.g. Find mobile broad band (demo)

24. Results: Scores per task Tool: Treejack

25. Is the new
structure
better?
Can customers find
products more easily?

26. Main results

27. 40% higher
success percentage
Main results

28. Deeper analysis of the results Tool: Treejack

29. Quick comparison of navigation
structures
Redesign easily communicated
with stakeholders
Results
Commitment to the new redesign
during design and development.

30. Case study 2 – Philips
Measure expected success of new concept

31. Problem
To which site
do I have to go?
user product(s)

32. Changes have high
impact on back-end
Is it worth the
investment?

33. http://upload.wikimedia.org/wikipedia/commons/thumb/4/45/Work_tools.jpg/1280px-Work_tools.jpg

34. Input: prototype of the website

35. Test setup
Current site Proposed solution
Prototype 115 pages 60 pages
Journeys
5 journeys, 5 journeys,
80 tasks 56 tasks
Respondents 800 respondents 600 respondents
Completed tasks 2400 completed tasks 1800 completed tasks

36. Please buy a Sensotouch 3D RQ1285/22. Please take note where you can buy this product
(online via Philips.com, online via a third party, or in a physical shop via a retailer).
Start task
Tool: Userzoom
Task: e.g. Buy a Sensotouch 3D Tool: Userzoom

37. Task: e.g. Buy a Sensotouch 3D (demo) Tool: Userzoom

38. Questions (satisfaction, ease of use) Tool: Userzoom

39. Results per task Tool: Userzoom

40. http://upload.wikimedia.org/wikipedia/commons/a/a5/Bundesarchiv_Bild_183-1990-0321-008%2C_Analyse_von_Wasserproben.jpg
Data analysis

41. Changes have high
impact on back-end
Is it worth the
investment?

42. Main results

43. +nn% -nn% +nn%
Goal completion Time to complete nn% increase
nn% increase nn% decrease in satisfaction
Main results

44. Detailed results

45. Screenflows Tool: Userzoom

46. Screenflows Tool: Userzoom

47. Heatmaps Tool: Userzoom

48. Realistic comparison
Convincing results to support an
important business decision
Results
First improvements on the concept
before development

49. Summary

50. understand conceive design implement optimize
web web
analytics analytics
Collect (new) data

51. Use quantitative analyses in the
design phase
Objective data facilitates
communication with stakeholders
takeaways Benchmark your design
It also helps identify issues before
development starts

52. Go and do this for yourselves!
takeaways

53. thank you!
www.userintelligence.com
@jnieuwland @martijn_vanloon #stakeholderpersuasion