Analytics to strengthen your UX process

1. Making Your UX Process
Effective and Persuasive
with Web Analytics
O’Neil | Rahul

2. Somesh Rahul
@SomeshRahul
Daniel O’Neil
@phoenix1189
Namaste! We’re Information Architects at
The Understanding Group (TUG)

3. How Somesh Got Here

4. How Daniel Got Here

5. Intended Takeaways
1. Be able to apply a framework for balancing quantitative
and qualitative research methods.
2. Have a grasp of several key Google Analytics tools that
are most relevant to UX practices.
3. Learn through labs, workshops and case studies how web
analytics is applied to actual UX projects.

6. Why Web Analytics

7. What Web Analytics Can and Can’t do

8. Analytics track actions, not intent!

9. Web Analytics and User Experience
Behaviors can infer intent
Quantitative guides Qualitative
Search Analytics for Your
Site
by - Lou Rosenfeld

10. The Web Analytics Framework

11. Business Goals and User Needs
Market, Audience, Seasonality
Testable On-Site Behavior
Website Analytics Works Best When They
are Measuring the Distillation of True Value

12. Business Goals /
User Needs
Website Goals
Described by:
Hypotheses
Market
Audience
Seasonality
Constrained and Organized by:
Filters, Segments, Time
Testable On-Site
Behavior
Tested by:
Descriptive
Analytics
Flow, Page Navigation, Nonstatistical narratives
Statistical
Analytics
A/B testing of Page Variations and Dimension
Segments

13. Goals and Hypotheses

14. Business Goals /
User Needs
Website Goals
Described by:
Hypotheses

15. Websites Run on Goals
A “Goal” is a measurable outcome resulting in a user
completing some desired activity on your website. Typical
goals are:
– Confirmation page at the end of a sales transaction.
– Thank-you page after filling out a contact or quote request form.
– Application or content downloads.
– Playing a game or watching a video on a site.

16. The Best Goals are Existentially Critical
– Is it a “Holiday Bonus” question?
– If this goal stopped happening, would your organization (or your
department) still exist?
– Most companies should have a few goals filtered through many
segments.

17. BUT...Goals Can’t Say Why or How
– Goals describe behavior badly.
– Goals can’t describe intent at all.

18. Hypotheses Link
Goals, Behavior,
and Theories
about Intent.

19. Hypotheses
A hypothesis suggests a functional change based on a
theory of action that has a measurable outcome.

20. Goals and Hypotheses
– Goals link clear up or down numbers to the outcome of
a specific site behavior.
– Hypotheses provide the testable narrative about how
the user’s experience on the site affects those goals.
– The testable narrative does not have to BE a goal, but
should specifically be IN SERVICE OF a goal.

21. Filters and Segments

22. Market
Audience
Seasonality
Constrained and Organized by:
Filters, Segments, Time

23. User Segments
shops like
consumer
designer
involved
in specifying
VIP at
current
contract
customer
VIP at
contract
prospect
?investor

24. Problems User Segments Addresses

25. Problems User Segments Addresses

26. Filters
- Todo: something about filters here
5
minutes

27. Descriptive Analytics

28. Testable On-Site
Behavior
Tested by:
Descriptive
Analytics
Flow, Page Navigation, Nonstatistical
narratives
Statistical
Analytics
A/B testing of Page Variations and Dimension
Segments

29. User Flows
- Structure User Interviews
- Create User Journey
- Find Path of Least Resistance

30. Stand-alone
Integrated
External
/ Social

31. Statistical Analytics

32. Testable On-Site
Behavior
Tested by:
Descriptive
Analytics
Flow, Page Navigation, Nonstatistical narratives
Statistical
Analytics
A/B testing of Page Variations and
Dimension Segments

33. What is Statistically Significant?
Determining whether the differences seen in data is more
than random chance.

34. Why Use It?
- Addresses the HiPPO problem.
- Saves time by getting to outcomes faster.
- Uncovers subtle effects.
- Confronts our own biases about aesthetic and design.

35. Quantifying a “measurable outcome”
If goals have been set up properly, outcomes can be
measured using simple statistics. And simple is all we need!
The recommended statistical method for UX professionals is
the A/B test.

36. Appropriate A/B Tests Should:
- Be immediately apparent to anyone looking comparing
the pages.
- Be defined in a functional UX way.
- Represent a set of coherent conceptual changes against
a single hypothesis.

37. Typical A/B Test Candidates
Question Testing For Best Testing Tool
Is the navigation layout
affecting conversion rate?
Conversion rate
by template
Google Analytics Experiements (Not out of
the box but you can hack it)
Which of two landing pages
performs better?
Conversion rate
by page version
Google Analytics Experiments
Which User segment
converts better
Conversion rate
compared by
User segment
Advanced segments, Confidence Interval
test

38. What Statistics Don’t Tell You
- Why a test failed. This can be just as critical as a
success.
- Why it succeeded.
- How to thoughtfully create testable hypotheses.

39. Marrying User Experience &
Web Analytics

40. Your UX Process

41. Abstracted UX Process
Discovery
Research and
Analysis
Design and
Testing

42. Discovery
Discovery
● Establish clearly the “Why” and
“Who” for the site.
● Organizational goals are
articulated and prioritized.
● The audience is clearly identified.
● The ultimate measures of success
are agreed upon.

43. Research and Analysis
Research and
Analysis
● Research how your users
approach your current site.
● Evaluate the website’s design and
information architecture.
● Synthesize the details into high-
level models that represent both
user needs and a high-level
information architecture.

44. Design and Testing
Design and
Testing
● Specify the site structure.
● Determine the ways in which the
goals will be achieved through site
structure.
● Test.

45. Qualitative and Quantitative Research
Discovery
Research and
Analysis
Design and
Testing
Qualitative
Research
(UX)
Quantitative
Research
(WA)
Stakeholder Interviews
Intention Modeling
User Interviews
Personas
User Journeys
Prototypes
Live Testing
Hypothesis Generation
Goal Context
User Segments
User Flows
A/B Tests

46. Thank You!