***This version of the presentation is for the eMetrics Summit in SF, April 17 2013. 11:15am in Room: Salon 7*** Chris and Farris expose the differences between how user experience designers and analytics practitioners think. While UXD weave best practices and user research into their designs, digital analysts spend their time confirming or refuting hypotheses in a data driven way. One approach is decidedly qualitative, the other decidedly quantitative. Learn how it is possible to leverage both enlightened design and deep data to continuously optimize user experiences. If you work on either side of this debate, this is how to better state your case… and get along with the other side.

1. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
UXD v. Analytics
What have YOU got for me?
No, what have you got for ME?
eMetrics Summit
San Francisco, CA| April 17th 2013
11:15am, Room: Salon 7
Chris Farnum, Senior User Experience Designer, ProQuest
Farris Khan, Analytics Lead, ProQuest & BobbleheadGuru blogger
1

2. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Dramatis Personae
Digital Analytics Lead..…Farris UXD/IA………………………..Chris
2

3. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Introduction – UXD v. Digital Analytics
Where do these specialties overlap?
How are the roles different?
Why should they collaborate?
Got a question or comment?
Tweet: @BobbleheadGuru or @crfarnum
Hashtags: #eMetrics #UXD
3

4. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
The ProQuest Search Platform
4

5. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
5

6. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Facets of the User Experience
From User Experience Design by Peter Morville
6

7. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
What is “User Experience Design”
From The Elements of User Experience by Jesse James Garret
7

8. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Ecosystem of Information Architecture
Business goals
Brand strategy
Technology
Politics
Resources
Context
Document types IA
Apps / Functionality
Content Objects
Volume
Content Users Audience
Structure Personas
Tasks
Needs
Info seeking behavior
Based on Rosenfeld and Morville – Information Architecture for the World Wide Web
8

9. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Elements of IA/UXD
• Organization Systems How is content organized? By User
Task, Subject Category, Audience Type, Metaphor…
• Labeling Systems Links, buttons, labels, calls-to-action…
• Navigation Systems Sitewide navigation, local navigation,
utility navigation…
• Search Systems Site search interfaces, results pages,
indexing…
• Thesauri, Controlled Vocabularies, and Metadata Systems of
terms used to organize and tag content
Based on Rosenfeld and Morville – Information Architecture for the World Wide Web
9

10. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Wireframes and Specifications
FIG 2: My Saved Articles FIG 2 Notes:
1. Page title
2. Count of all items in the list.
• Increments as items added
1 • Decrements as items deleted
2 4
3. Link back to last set of search results
3
5 4. Sort options:
• By date added – reverse chron
• By date published – reverse chron
• Alphabetical by title
6 5. Checkbox to select all items in the list
• Checking selects all items
• Unchecking deselects all items
6. Articles. Each item includes:
• Checkbox
• Number in list
• Citation – in same style as in
search results
• Date added – DD Mon YYYY
10

11. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
The UXD Role at ProQuest
Core UXD
Design Processes
Prioritized Develop/Test
requirements
Planning
Iteration release Product
release
11

12. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
The UXD Approach, In General…
12

13. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
A newcomer to the department…
13

14. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Overlapping Territories? Competition?
14

15. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
What is Digital Analytics?
15

16. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Ecosystem of Analytics
Object of Analysis:
Document, Library, Sy
stem
Metrics:
What?
Usage
Object of Analysis: Object of Analysis:
Visitor Attitudinal/ Behavioral/ Visitor
Metrics: Persona Observational Metrics:
Who? Why? Where? When?
Common Unit of Analysis
16

17. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Monthly Analytics Heatmap
Notes:
1. Data from March 2012 through February 2013.
2. Source: WebTrends and Internal data collection.
17

18. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Visits: Year-Over-Year Comparison
18

19. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Visits/Visitors
19

20. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Weekly Analytics Heatmap
20

21. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Year Over Year Visits Per Week
Thanksgiving (US)
Easter
Christmas
21

22. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Daily Analytics Heatmap
22

23. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Visits per Day
(Since February 13, 2011)

24. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Percentage of Visits with Error Pages
Notes:
1. Data from January 1, 2012 through March 17, 2013.
2. Source: WebTrends
3. Four “Spike Days”: Feb. 1, 2012, March 17, 2012 and March 1, 2013.
24

25. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Percentage of Visits with Error Pages
Notes:
1. Hour-By-Hour Analysis of March 1, 2013.
2. Source: WebTrends
Analysis:
1. This event occurred over a two hour period (10am through 12noon)
25

26. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Charts directly from tools can look… well…
26

27. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Like Uncooked Spaghetti
27

28. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Just one strand…
28

29. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Confidence Interval Analysis
29

30. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
80% Interval Analysis
30

31. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Increasing Confidence - 80%, 90%, 95%
31

32. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
The DA Approach, In General…
32

33. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Conversation #1 – three weeks later…
33

34. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
It’s monthly report day, and Farris is looking hard at the data…
34

35. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Farris’ Question
“Why do some of our segments have
significantly higher satisfaction scores
than others?”
35

36. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
80% Interval Analysis with Narrow Scale
36

37. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
1st Order Report
37

38. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
2nd Order Report
38

39. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
3rd Order Report: NO High School or Librarians
39

40. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
UXD Clue: Persona Driven Design
40

41. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Interpreting Satisfaction Data
I. 2nd year students are intermediate
researchers, therefore have highest
satisfaction.
II. 3rd year students are advanced
searchers and more likely to need
advanced search tools. Satisfaction
drops in 3rd year, so they may need
more attention.
41

42. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
End of scene
42

43. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Conversation #1 – Summary
DA offers intelligence
via segment analysis
UXD offers context
via personas
43

44. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Conversation #2 – two weeks later…
44

45. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
One morning Chris opened his email to find…
45

46. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Chris’ Question
“There’s a customer service request to
improve the ease of narrowing results.
Is there any user data that can give me
the background I need to design some
new approaches?”
46

47. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
UXD: Do we need more options, less,…?
47

48. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
DA Clue: Satisfaction Summary
48

49. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Model Question Summary
Higher Scores 
Legend:
Accuracy of Information (1.2, 8.1)
(Impact, Score)
Quality of Information (1.2, 8.1)
Breath of Content (1.2, 8.0)
How Quickly Pages Load (0.5, 7.9)
Consistency of Speed Page to Page (0.5, 7.9)
Ability to Load Pages without Errors (0.5, 7.9)
Readability of Graphics (0.5, 7.9)
How Well Site Is Organized (1.7, 7.5)
Balance of Graphics and Text (0.5, 7.3)
Area of Focus
How Well Site Layout Helps Find (1.7, 7.2) Relevance of Search Results (2.8, 7.2)
Organization of Search Results (2.8, 7.2)
How Well Search Results Help Selection (2.8, 7.2)
Options for Navigation (1.7, 7.1)
How Well Search Options Help Narrow Results (2.8, 7.0)
Visual Appeal (0.5, 6.9) Number of Clicks (1.7, 6.9)
Higher Impact 
49

50. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Open Ended Comments With Respondent Level Data
Narrow down their searches to
more specifics
Provide Tips for narrowing
Search
Provide more user friendly
narrowing devices
Be able to narrow search results
to only find articles with full text
50

51. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Confirmed…
I. Comment was not an outlier.
II. Improving narrowing options
should significantly impact
satisfaction.
51

52. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
End of scene
52

53. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Conversation #2 – Summary
UXD needs background
info before design.
Analytics offers
quantitative 
qualitative data
53

54. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Conversation #3 – four weeks later…
54

55. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Farris’ Question
“I was preparing my monthly
report and noticed something
new. Page views per visit
dropped, but other statistics
remained constant. What does
this mean?”
55

56. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
DA: Weekly Heatmap
56

57. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
UXD: Updates to site navigation, database selection
57

58. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Answers and More Questions
I. Hypothesis - dynamic page
element and/or other design
changes caused page counts to
decrease.
II. No increase in satisfaction… yet.
Further adjustments needed?
58

59. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
End scene…
59

60. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Conversation #3 – Summary
DA spots changes efficiently.
UXD can offer hypotheses.
60

61. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
UXD v. Analytics - Differences
Digital Analytics User Experience Design
Quantitative Qualitative
“What did most users do?” “Why did you do that?”
Reporting trends Design decisions
Broad & shallow view Deep & narrow view
Big data analysis Live user research
61

62. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
UXD & Analytics - Common Goals
Digital Analytics User Experience Design
Deep interest in user behavior
Want to increase user satisfaction
Desire to influence strategic decisions
By leveraging each other….
Does 1 + 1 = 3?
62

63. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Working Better Together
• Give UXD access to analytics tools
• Project collaboration
• Refine survey questions and KPIs
• Design informed by data insights
• Deeper analysis based on a richer understanding of user
personas and motivation
• Smarter user research – finding respondents, asking better
questions.
63

64. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD
Questions and Comments
Farris Khan
Farris.Khan@ProQuest.com
@Bobbleheadguru
Chris Farnum
Chris.Farnum@ProQuest.com
@crfarnum
Slideshare
http://www.slideshare.net/ChrisFarnum/
64