The document provides an overview of web metrics and user experience (UX) metrics for benchmarking and evaluating websites. It discusses key metrics categories including traffic, engagement, audience, and platform. Common metrics for each category like unique visitors, pageviews, and bounce rate are explained. The document also outlines an 11-step process for conducting a competitive metrics analysis, which includes identifying goals, collecting competitor data, analyzing metrics, and generating findings. Examples of metrics analyses for time spent, bounce rate, and demographics are presented.
1. UX + WEB METRICS
OXFORD TECHNOLOGY
VENTURES
How to Benchmark, Measure
& Evaluate UX Impact
!
!
General Assembly
August 03, 2014
Bev May
Beverly@OxfordTech.us @OxfordTech OxfordTech.us
@UXAwards UXAwards.org
2. QUESTIONS
OXFORD TECHNOLOGY
VENTURES
1. Have own Project / Sector to Research?
2. Web-Based?
3. Bring Metrics in a Spreadsheet?
4. Have a Computer?
5. Level of Familiarity with Web Metrics?
6. Any Data Scientists / Statisticians / Metrics Pros?
7. # Years Experience in Tech / Digital?
3. OxfordTech.us | GA: Web Metrics l August 2014
http://rlv.zcache.com/im_right_youre_wrong_next_coffee_mug-rbaffc7f99bad4d23bca7de839ac40bff_x7k28_8byvr_512.jpg
4. March to Nowhere
NO $$
NO TIME
WE KNOW OUR INDUSTRY
WE KNOW WHAT OUR AUDIENCE WANTS
AS AN EXPERT- YOU SHOULD KNOW WHAT’S BEST
STIFLE CREATIVITY / FUEL MEDIOCRITY
OxfordTech.us | GA: Web Metrics l August 2014
5. Benefit 1: THE UNKNOWN UNKNOWNS.
OxfordTech.us | GA: Web Metrics l August 2014
Donald Rumsefeld, wikipedia.com
6. OxfordTech.us | GA: Web Metrics l August 2014
Munch, the Scream- Wikipedia.org/
+ Improve Outcomes
7. OxfordTech.us | GA: Web Metrics l August 2014
LEAN: Helps Validate
Before Dev & Launch
(not just after!)
http://static.guim.co.uk/sys-images/Guardian/Pix/pictures/2011/8/22/1314031516692/A-boy-jumps-off-a-diving--007.jpg
8. OxfordTech.us | GA: Web Metrics l August 2014
WOW!
http://static.guim.co.uk/sys-images/Guardian/Pix/pictures/2011/8/22/1314031516692/A-boy-jumps-off-a-diving--007.jpg
9. Today
PART 1: OVERVIEW & CASE STUDY 1.5 HOURS
▪ CONTEXT & METRICS OVERVIEW
▪ PROCESS & IN-DEPTH CASE STUDY EXAMPLE (11 STEPS)
▪ TOP 10 NEWBIE MISTAKES
▪ SUMMARY & Q+A
!
BREAK
!
PART 2: WORKSHOP!
▪ DATA VIZ CHOICES
▪ DEFINE YOUR KPIS
▪ RESEARCH & GRAPH DATA OF COMPETITORS
POSSIBLE 2nd BREAK
!
▪ ANALYZE RESULTS & SHARE
▪ IF TIME - KPI-FOCUSED COMPETITIVE ANALYSIS
▪ RESOURCES
OxfordTech.us | GA: Web Metrics l August 2014
11. OxfordTech.us | GA: Web Metrics l August 2014
What is UX?
http://etc.usf.edu/clipart/70500/70542/70542_264_ra-090_o_lg.gif
12. User Centered Design
AUDIENCE
!
CUSTOMER PROBLEM
!
RESEARCH
!
PROTOTYPE
!
VALIDATE & EVALUATE
!
ITERATE
OxfordTech.us | GA: Web Metrics l August 2014
13. Holistic Design Thinking
Assumptions
true? How to
improve?
OxfordTech.us | GA: Web Metrics l August 2014
Who is your
customer?
What’s
their problem?
What’s
your
solution?
How is it unique/
better than
current?
14. Lean Startup = Good UCD / UX
MEASURE
LEARN
IDEA/ BUILD
OxfordTech.us | GA: Web Metrics l August 2014
19. UX Awards
Premier awards for exceptional digital experience, now in its 4th year
OxfordTech.us | GA: Web Metrics l August 2014
UXAWARDS.ORG
20. UXies: Data Driven Judging
OxfordTech.us | GA: Web Metrics l August 2014
UXAWARDS.ORG
21. Be a Competent UX Generalist
OxfordTech.us | GA: Web Metrics l August 2014
http://lawyerkm.com/wp-content/uploads/2008/09/swiss_army_knife1.jpg
Metrics
UX
23. What are Metrics?
#%<>
OxfordTech.us | GA: Web Metrics l August 2014
24. UX Research Methods
• HALLWAY USABILITY
• OBSERVATION
• INTERACTIVE TESTING (EYE TRACKING, ETC.)
• HEAT MAPS
• CARD SORTS
• SURVEYS
• HEURISTIC EVALUATIONS
• MARKET RESEARCH
• PUBLIC METRICS RESEARCH
• METRICS ANALYSIS
• MVT & A/B
OxfordTech.us | Web Metrics l July 2014
http://www.dsr.wa.gov.au/assets/images/Diagrams/Darts-playing-area.gif!
25. Metrics Data Sources
▪ Web- public & competitor
▪ Web- internal / private
▪ Social
▪ Mobile Apps
▪ User Testing & Analysis
▪ MVT & A/B
▪ Surveys
▪ Ecommerce
OxfordTech.us | GA: Web Metrics l August 2014
26. Metrics Categories
TRAFFIC
!
ENGAGEMENT
!
AUDIENCE
!
PLATFORM
!
(REVENUE)
OxfordTech.us | GA: Web Metrics l August 2014
27. Key Traffic Web Metrics
MARKETING FOCUS
▪ Uniques
▪ Visits
▪ % from Search -Paid vs. Organic,
Top Referring Terms
▪ % from Social
▪ % direct-load
▪ Top Referring Domains
OxfordTech.us | GA: Web Metrics l August 2014
28. Key Engagement Web Metrics
UX FOCUS
▪ Visits/ Unique
▪ Page Views (PV)
▪ PVs/Visit, PVs/ Unique
▪ Time Spent
▪ Bounce Rate (1 page/ visit)
▪ Top Entry / Exit Pages
▪ Top Sub-Sites/ Sections
OxfordTech.us | GA: Web Metrics l August 2014
http://blog.hugeaim.com/static/wp-content/uploads/2011/07/ballbounce.jpg
29. Key Audience Web Metrics
▪ Demographics- Age, Income,
Gender, Education, Location,
Ethnicity, Marital Status, Kids
OxfordTech.us | GA: Web Metrics l August 2014
http://clipartist.info/openclipart.org/SVG/rejon/person_outline_4-800px.png
30. Key Platform Web Metrics
DESIGN FOCUS
▪ % Mobile
▪ Display size & resolution
- desktop & mobile
▪ OS, Device, Web Speed
OxfordTech.us | GA: Web Metrics l August 2014
http://www.gizmoville.com/wp-content/uploads/2012/02/omgitsfullofpixels.png
31. Summary - Public Web Metrics
TRAFFIC- MARKETING
▪ Uniques
▪ Visits
▪ % from Search -Paid vs. Organic,
Top Referring Terms
▪ % from Social
▪ % direct-load
▪ Top Referring Domains
▪ Bounce Rate (1 page/ visit)
!
AUDIENCE - EVERYONE
▪ Demographics- Age, Income,
Gender, Education, Location,
Ethnicity, Marital Status, Kids
OxfordTech.us | GA: Web Metrics l August 2014
ENGAGEMENT- UX
▪ Visits/ Unique
▪ Page Views (PV)
▪ PVs/Visit, PVs/ Unique
▪ Time Spent
▪ Bounce Rate (1 page/ visit)
▪ Top Entry / Exit Pages
▪ Top Sub-Sites/ Sections
!
PLATFORM- DESIGN
▪ % Mobile
▪ Display size & resolution - desktop
& mobile
▪ OS, Device, Web Speed
!
!
32. Questions?
TRAFFIC- MARKETING
▪ Uniques
▪ Visits
▪ % from Search -Paid vs. Organic,
Top Referring Terms
▪ % from Social
▪ % direct-load
▪ Top Referring Domains
▪ Bounce Rate (1 page/ visit)
!
AUDIENCE - EVERYONE
▪ Demographics- Age, Income,
Gender, Education, Location,
Ethnicity, Marital Status, Kids
OxfordTech.us | GA: Web Metrics l August 2014
ENGAGEMENT- UX
▪ Visits/ Unique
▪ Page Views (PV)
▪ PVs/Visit, PVs/ Unique
▪ Time Spent
▪ Bounce Rate (1 page/ visit)
▪ Top Entry / Exit Pages
▪ Top Sub-Sites/ Sections
!
PLATFORM- DESIGN
▪ % Mobile
▪ Display size & resolution - desktop
& mobile
▪ OS, Device, Web Speed
!
!
33. How Public Web Metrics Work
!
Statistical Samples
▪ JavaScript
▪ Cookies
▪ Pixels
▪ Server-side tracking
▪ Web Traffic
Public = Inaccurate
▪ Won’t be listed on public metrics sites if too small/ new
▪ Heed the warnings
OxfordTech.us | GA: Web Metrics l August 2014
http://www.wsgsystems.com/uploads/images/cookies_large.jpg
34. Internal Metrics
!
▪ Usually more accurate
▪ Requires at least some development
▪ Little competitor visibility (unless high-cost)
▪ Click path Analysis
▪ Heat map Analysis
▪ % Logged In/ Out
▪ Ecommerce: ARPU/RPC, R/T, R/V
▪ Data by Sections/ Categories
▪ Top & Bottom Performing Pages, Sections
▪ Top Entry/ Exit Pages, Sections- More Detailed
▪ Top On-Site Search Terms, 404 pages
▪ % Mobile by Page/ Section/ category
▪ Demographics- Politics, Interests, Credit, Job,
OxfordTech.us | GA: Web Metrics l August 2014
Title
http: //www.damenationblog.com/wp-content/uploads/2012/06/iStock_000019717637Smal l . jpg
36. Client Case Study, Jan. 2014
TV/ DIGITAL MEDIA BRAND ANALYSIS- OUTPUTS
A. Competitive Metrics Analysis (PPT, 80 slides)
+ Excel Spreadsheet - 24 Competitors, 5 Categories
B. UX Competitive Heuristic Evaluation (PPT, 134 slides)
C. Quantified Summary of Competitor Evaluation
D. Internal Site Metrics Analysis (PPT, 82 slides)
E. Internal UX Site Evaluation (PPT, 114 slides)
F. New UX Concepting
OxfordTech.us | GA: Web Metrics l August 2014
37. 11 Steps
1. Identify key sites, goals & KPIs
2. Get competitor metrics data from multiple sources
3. Graph data after standardizing in Excel
4. Check for oddities
5. Don’t be a Robot. Review, Analyze & THINK
6. Examine High Performers (Heuristic, Quantified)
7. Analyze Internal Site Metrics
8. Review Internal Site (Heuristic)
9. Generate Actionable Findings
10. Concept New Directions
11. Test & Iterate
OxfordTech.us | GA: Web Metrics l August 2014
38. Data Sources Used: MANUAL Process
INTERNAL
1. Comscore
2. Nielsen
3. Adobe Site Catalyst
!
EXTERNAL
1. Quantcast
2. Alexa
3. Compete
4. SimilarWeb
!
SOCIAL
1. Facebook
2. Twitter
3. YouTube
4. Klout
!
MOBILE
1. iTunes
2. Google Play
OxfordTech.us | GA: Web Metrics l August 2014
40. BUSINESS KPI (RED) & UX (BLACK) Goals
< VIDEO
DISCOVERY
OxfordTech.us | GA: Web Metrics l August 2014
< SOCIAL
VISITS
SEE < FULL
VIDEOS
< BRAND
AWARENESS
41. Define Your Metrics KPIs
Benchmarks should be similar by industry, sector & objective, BUT each
project will differ
EXAMPLES
ECOMMERCE ARPU or RPV, R/T, R/V
MARKETING CTR, VIEWS, CONVERSION RATE
VIDEO TIME SPENT, # VIDEO VIEWED/VISIT,
# REPEAT VISITS, UNIQUES
PUBLISHING PV/V, V/UNIQUE, (TIME SPENT)
AD-DRIVEN IMPRESSIONS, MONTHLY PVs, PV/V
SOCIAL SHARES, LIKES, FAVES, FOLLOWS
OxfordTech.us | GA: Web Metrics l August 2014
42. Quantify Your Goals
3+ SHORT VIDEOS/
SESSION ACROSS
MULTIPLE SHOWS
(VIDEO DISCOVERY)
OxfordTech.us | GA: Web Metrics l August 2014
GET 30%
TRAFFIC
FROM
SOCIAL
43. Translate into UX Characteristics
< CONTENT DISCOVERY
< BROWSE
< SEARCH
< AUTO-SUGGEST
< NAVIGATION
OxfordTech.us | GA: Web Metrics l August 2014
< SOCIAL
ENGAGEMENT &
SHARING TOOLS
< SOCIAL WIDGETS
44. OxfordTech.us | GA: Web Metrics l August 2014
http://www.aguntherphotography.com/files/images/1833_large.jpg
You Must Choose.
45. Consensus - Focus & Prioritize
OxfordTech.us | GA: Web Metrics l August 2014
53. Traffic (Visits) Over Time - Subset
Visits/Time
OxfordTech.us | GA: Web Metrics l August 2014
54. Uniques/Time – 1 year (Subset)
30,000,000
22,500,000
15,000,000
7,500,000
0
Dec-‐16 Jan-‐17 Feb-‐17 Mar-‐17 Apr-‐17 May-‐17 Jun-‐17 Jul-‐17 Aug-‐17 Sept-‐17 Oct-‐17 Nov-‐17
OxfordTech.us | GA: Web Metrics l August 2014
Bleacher Report
Vimeo
TMZ
Gawker
The Onion
History
Comedy Central
Discovery
A&E
Funny or Die
Mashable
Boing Boing
Engadget
TruTV
DailyMoRon
55. Spreadsheet: Traffic Over Time
35 Competitors. Documented Source, Metric, Date Ranges, Date, Average
Below: Uniques/mo over time
OxfordTech.us | GA: Web Metrics l August 2014
69. 4. REVIEW FOR
WEIRDNESS &
CASVENTURES
5.
ANALYZE FOR
MEANING
70. Review the Data for Anomalies, Exceptions
▪ THINK HARD & CAREFULLY
▪ Cross-Check & Correlate
▪ Be able to defend
▪ Take special note of any warnings
OxfordTech.us | GA: Web Metrics l August 2014
71. The Case of Netflix
▪ 700 Million monthly visits
▪ 260 Million monthly visits
▪ 126.5 Million weekly visits
▪ 80 Million weekly visits
▪ 10 Million daily visits
▪ 12 Million monthly visits
▪ 8 Million monthly uniques?
OxfordTech.us | GA: Web Metrics l August 2014
72. How many trees?
OxfordTech.us | GA: Web Metrics l August 2014
?
77. OxfordTech.us | GA: Web Metrics l August 2014
Borrow from the Best
h t tp: //i m a g e s 4 . fanpop.com/image/p h o to s / 14700000/S e e -n o - ev i l -h e a r-n o - ev i l - sp e a k-n o - ev i l -monkey s-14750406 -1600-1200. jpg
79. Competitor Heuristic Evaluation
▪Deep-dive into high-performers- how & why
▪Correlated to the high-performing sites from metrics
OxfordTech.us | GA: Web Metrics l August 2014
81. Try to find out the WHY behind the #s
OxfordTech.us | GA: Web Metrics l August 2014
82. Quantify Your Heuristic Analysis
Social Features Considered:
• Easy to find content based on tagging?
• Easy to follow/receive notification of upcoming shows?
• User ratings or comments?
• Save a show with login?
• Easy to follow/watch/share on social media?
!
Social-Top Features:
• Clips organized by
content tag
• “Follow” content
• Account profile with
saved shows
OxfordTech.us | GA: Web Metrics l August 2014
83. Quantify Your Heuristic Analysis
Video Player-Best Experience:
Hulu, CBS, MTV, Amazon
Instant Video
Video Player- Top characteristics include:
• Episode/series description obvious to user
• Previews, clips, and recommended content available
under or next to player
• Social Media sharing in player
OxfordTech.us | GA: Web Metrics l August 2014
87. Internal Metrics: A Deep Dive
http://346fae7859434bda978f-‐1902f231618c5697bb2c852c565827b8.r12.cf5.rackcdn.com/wp-‐content/uploads/2014/02/diving1.jpg
OxfordTech.us | GA: Web Metrics l August 2014
95. Key Internal Site Metrics Over Time
OxfordTech.us | GA: Web Metrics l August 2014
96. Internal Site Metrics – Cohort Analysis
▪ Lean Startup / Agile technique
▪ Tracks KPIs per sprint/ at fixed time blocks to evaluate if there are
improvements from new product releases
http://idea-stack.blogspot.com/2013/04/quick-hack-setting-up-cohort-analysis.html
OxfordTech.us | GA: Web Metrics l August 2014
97. Internal Site Metrics- Mobile vs. Desktop
Web vs. mobile activity by Site Section
OxfordTech.us | GA: Web Metrics l August 2014
109. SEXY IT UP!
OxfordTech.us | GA: Web Metrics l August 2014
http://tng.trekcore.com/hd/albums/1x13/datalore_hd_027.jpg
110. Snoozeville
OxfordTech.us | GA: Web Metrics l August 2014
http://m.cdn.blog.hu/in/investo/image/Robot.jpg
http://cailincreature.blogspot.com/2010_06_01_archive.html
111. Don’t Make Me Think!
OxfordTech.us | GA: Web Metrics l August 2014
http://cailincreature.blogspot.com/2010_06_01_archive.html
112. Risk of Wrong Conclusions
OxfordTech.us | GA: Web Metrics l August 2014
http://cailincreature.blogspot.com/2010_06_01_archive.html
113. The path forward - based on data
OxfordTech.us | GA: Web Metrics l August 2014
http://m.cdn.blog.hu/in/investo/image/Robot.jpg
114. KPI Impact is SEXY
$
:-)
OxfordTech.us | GA: Web Metrics l August 2014
115. Analogy: UX Awards
Youtube.com/UXAwards
OxfordTech.us | GA: Web Metrics l August 2014
117. New UX Ideation Guided By KPIs, Metrics
UX GOALS BASED ON PRODUCT KPIS
- BUSINESS: Increase # full video views/session (with prerolls)
- USER: Find & watch videos, fast (in under N clicks/ X seconds)
http://www.sonicftp.com/news/images/guitarstorage_carousel.jpg
OxfordTech.us | GA: Web Metrics l August 2014
118. Example UX Ideation
ENGAGEMENT KPIS
- # Video Views
!
IDEATION: CONTENT DISCOVERY
- What are all the ways to browse or find similar content?
- What are all the ways to discover new content?
- “What if”….?
- What are successful competitors / “admirables” doing?
- Offline analogies?
http://www.sonicftp.com/news/images/guitarstorage_carousel.jpg
OxfordTech.us | GA: Web Metrics l August 2014
119. Seeking Inspiration Offline
OFFLINE SHOPPERS: BROWSE BY BRAND OR FUNCTIONALITY
ONLINE: BY SHOW, ACTOR, CHARACTER OR GENRE, LENGTH
thefashionspot.OxfordTech.us com/| GA: shop/Web 364135-Metrics boutique-l August of-the-2014
week-the-chanel-shop-at-bergdorf-goodman/
121. Getting Creative- Summary
▪ ASK “WHAT IF” / “WHAT ARE” - FROM KPIS & TARGET ACTIONS
▪ LIST ALL THE POSSIBLE OPTIONS
▪ CONSIDER RELEVANCE LAST
OxfordTech.us | GA: Web Metrics l August 2014
http://openclipart.org/image/2400px/svg_to_png/185271/ico_light_bulb_2.png
122. KPIs, Goals Can Conflict (UX must Solve)
Final direction may need to balance competing needs
!
Ecommerce
▪ Business: maximum revenue/ visitor or /transaction
▪ Customer: cheapest, best option that’s found the most
easily (least clicks & time on site before transaction)
Publishing (ad-driven)
▪ Business: maximum ad impressions & PVs/visit
▪ Visitor: least ads, clicks & PVs to find & consume desired
content
OxfordTech.us | Web Metrics l July 2014
124. Testing & Iterating Your Concepts
GET FEEDBACK BY TESTING BEFORE DEV.
▪Clickable Prototypes
▪MVT - Landing Pages
USE METRICS TO EVALUATE LAUNCHED FEATURES
http://www.matraxis.co.uk/services/ab-multivariate-testing/
OxfordTech.us | GA: Web Metrics l August 2014
125. Testing & Iterating Your Concepts
OxfordTech.us | GA: Web Metrics l August 2014
ANDREW MCKINNEY!
http://andrewmckinney.com/projects/weight-watchers-iphone-app/
126. Review of the 11 Steps
1. Identify key sites, goals & KPIs
!
2. Get competitor metrics data from multiple sources
3. Graph data after standardizing in Excel
4. Check for oddities
5. Don’t be a Robot. Review, Analyze & THINK
!
6. Examine High Performers (Heuristic, Quantified)
!
7. Analyze Internal Site Metrics
8. Review Internal Site (Heuristic, also Quantified)
!
9. Generate Actionable Findings inside a narrative
10. Concept New Directions
11. Test & Iterate
OxfordTech.us | GA: Web Metrics l August 2014
128. Top 10 Newbie Metrics Mistakes
1. Not knowing what to look for, based on business & user goals
2. Using the wrong metrics & KPIs for the job
3. Believing any metrics are accurate – esp. public web
4. Being a robot. Think and be skeptical
5. Comparing across different time ranges
6. Using only 1 short time period; forgetting seasonality/ trends
7. Comparing across different sources with different names
8. Comparing internal to public metrics (correlation, not exact)
9. Not understanding why bad sites have good metrics.
10. Not asking WHY or HOW
DON’T BE A ROBOT.
http://m.cdn.blog.hu/in/investo/image/Robot.jpg
OxfordTech.us | GA: Web Metrics l August 2014
129. Using the Wrong Metrics
OxfordTech.us | GA: Web Metrics l August 2014
http://www.clker.com/clipart-10628.html
131. When Good #s Are Bad.
HIGH ENGAGEMENT CAN BE FROM TERRIBLE UX.
Poor usability can lead to high PV/u, Time Spent, Deep & wide clickpaths
OxfordTech.us | GA: Web Metrics l August 2014
http://www.1800attorney.com/
132. Humans > Data
OxfordTech.us | GA: Web Metrics l August 2014
134. Steps
OxfordTech.us | GA: Web Metrics l August 2014
http://m.cdn.blog.hu/in/investo/image/Robot.jpg
1. Identify key sites & KPIs
2. Get data from multiple sources
3. Graph it
4. Check for oddities
5. Don’t be a Robot. Review, Analyze & THINK
6. Examine High Performers (Heuristic, Quantified)
7. Analyze Internal Site Metrics
8. Review Internal Site (Heuristic)
9. Generate Actionable Findings
10. Concept New Directions
11. Test & Iterate
135. Web Metrics Are NOT…
▪Accurate
!
▪Absolutes
!
▪Useful when quoted in isolation or without analysis
!
▪Useful without understanding the project & context
OxfordTech.us | GA: Web Metrics l August 2014
136. Theory of Relativity
OxfordTech.us | GA: Web Metrics l August 2014
chican-izmo.blogspot.com/2010/06/if-tree-falls-in-forest.html
137. Web Metrics Are…
▪Useful approximate data points
!
▪Fantastic for ALL STAGES OF UX
!
▪A means to get UX out of the gallery of opinions by quantifying
the value of UX based on performance
!
▪ A way to pinpoint top competitors for further review, ideas
!
▪Great to narrow product goals & engagement KPIs
!
▪Helpful with ideation through goal-focused brainstorming
!
▪ A means to improve performance through multivariate testing
OxfordTech.us | GA: Web Metrics l August 2014
139. OxfordTech.us | GA: Web Metrics l August 2014
Show AND Tell the Story
lorenweisman.com/2013/06/21/music-marketing-plan/music-marketing-plan-storytime-artists-guide-show-and-tell/
145. Steps
OxfordTech.us | GA: Web Metrics l August 2014
http://m.cdn.blog.hu/in/investo/image/Robot.jpg
1. Identify key sites & KPIs
2. Get data from multiple sources
3. Graph it
4. Check for oddities
5. Don’t be a Robot. Review, Analyze & THINK
6. Examine High Performers (Heuristic, Quantified)
7. Analyze Internal Site Metrics
8. Review Internal Site (Heuristic)
9. Generate Actionable Findings
10. Concept New Directions
11. Test & Iterate
146. Web Metrics Are NOT…
▪Accurate
!
▪Absolutes
!
▪Useful when quoted in isolation or without analysis
!
▪Useful without understanding the project & context
OxfordTech.us | GA: Web Metrics l August 2014
147. Theory of Relativity
OxfordTech.us | GA: Web Metrics l August 2014
chican-izmo.blogspot.com/2010/06/if-tree-falls-in-forest.html
148. Web Metrics Are…
▪Useful approximate data points
!
▪Fantastic for ALL STAGES OF UX
!
▪A means to get UX out of the gallery of opinions by quantifying
the value of UX based on performance
!
▪ A way to pinpoint top competitors for further review, ideas
!
▪Great to narrow product goals & engagement KPIs
!
▪Helpful with ideation through goal-focused brainstorming
!
▪ A means to improve performance through multivariate testing
OxfordTech.us | GA: Web Metrics l August 2014
150. OxfordTech.us | GA: Web Metrics l August 2014
Show AND Tell the Story
lorenweisman.com/2013/06/21/music-marketing-plan/music-marketing-plan-storytime-artists-guide-show-and-tell/
163. Process Steps We’ll Do Today
1. Identify key sites, goals & KPIs
2. Get competitor metrics data from multiple
OxfordTech.us | GA: Web Metrics l August 2014
http://m.cdn.blog.hu/in/investo/image/Robot.jpg
sources
3. Graph data after standardizing in Excel
4. Check for oddities
5. Don’t be a Robot. Review, Analyze & THINK
6. If Time- Examine High Performers (Heuristic,
Quantified)
7. Analyze Internal Site Metrics
8. Review Internal Site (Heuristic)
9. Generate Actionable Findings
10. Concept New Directions
11. Test & Iterate
164. Workshop Agenda
1. Form Teams
2. Define Project KPIs
3. Web Metrics- Download & Graph
4. Find Analysis & Meaning
5. Share Findings
6. (If Time) KPI-Focused Competitive Analysis
7. Resources + Q&A
OxfordTech.us | GA: Web Metrics l August 2014
165. Form Teams
▪ Teams of 2-4
▪ BUT RECOMMEND EVERYONE DOES THE WORK (IN PARALLEL)
▪ Introduce yourself to your teams!
NEEDS:
▪ Computer with Excel / Spreadsheet Program for data work
▪ Choose a Web-Based Sector – with established metrics
▪ 5 Competitors / Comparables (known, large sites only)
▪ Default sectors to research if you don’t have any: Publishing or Video
OxfordTech.us | GA: Web Metrics l August 2014
166. 1. KPIs & Goals (Strategy)- 5-10 Mins
▪ Define Business Strategy (Revenue Model) & Business KPIS
▪ Define User Goals & Priorities
▪ Translate into UX Features, Goals
▪ Define Evaluative Metrics KPIs & Goals
▪ PICK 2-3 KEY WEB METRICS TO RESEARCH
▪ Sector & Sub-Sector
▪ Comparables
▪ “Admirables”
▪ PICK 5 SITES
OxfordTech.us | GA: Web Metrics l August 2014
167. BUSINESS (RED), USER (GREY) & UX (BLACK)
< VIDEO
DISCOVERY
OxfordTech.us | GA: Web Metrics l August 2014
< SOCIAL VISITS
SEE < FULL
(PAID) VIDEOS
< BRAND
AWARENESS
FIND + WATCH TOP
FREE CLIPS
EASILY SHARE
WITH FRIENDS
168. Define Your Evaluative Metrics KPIs
Benchmarks should be similar by industry, sector & objective
EXAMPLES
ECOMMERCE ARPU or RPV, R/T, R/V
MARKETING CTR, VIEWS, CONVERSION RATE
VIDEO TIME SPENT, # VIDEOS VIEWED/VISIT
PUBLISHING PV/V, V/UNIQUE, (TIME SPENT)
AD-DRIVEN IMPRESSIONS, MONTHLY PVs, PV/V
SOCIAL SHARES, LIKES, FAVES, FOLLOWS
OxfordTech.us | GA: Web Metrics l August 2014
169. Translate into Key UX Characteristics
< CONTENT DISCOVERY
< BROWSE
< SEARCH
< AUTO-SUGGEST
< NAVIGATION
OxfordTech.us | GA: Web Metrics l August 2014
< SOCIAL
ENGAGEMENT &
SHARING TOOLS
< SOCIAL WIDGETS
170. (Quantify Your Goals)
3+ SHORT VIDEOS/ SESSION
ACROSS MULTIPLE SHOWS
(VIDEO DISCOVERY)
OxfordTech.us | GA: Web Metrics l August 2014
GET 30%
TRAFFIC FROM
SOCIAL
171. KPIs & Goals (Strategy)- Share!
OxfordTech.us | GA: Web Metrics l August 2014
172. 2. Get Competitor Data into a Spreadsheet
▪ Download/ manually transcribe from many sources- never just one
▪ ALEXA: http://www.alexa.com/siteinfo/nameofsite.com
▪ COMPETE: https://siteanalytics.compete.com/nameofsite.com
▪ QUANTCAST: https://www.quantcast.com/nameofsite.com
▪ SIMILARWEB: http://www.similarweb.com/website/nameofsite.com
OxfordTech.us | GA: Web Metrics l August 2014
173. 2. Get Data, Put Into Spreadsheet
TODAY AS A GROUP- WEB METRICS
• Time Spent – Engagement (Bar Chart)
• Bounce Rate
• Demographics-Gender – Audience (Bar Chart, Pie Chart)
• Unique Visitors over Time – Popularity (Line Graphs)
• Comparisons- Bounce Rate
• Your KPIs
• Scatterplot Comparisons
OxfordTech.us | GA: Web Metrics l August 2014
174. A. Time Spent on Site- Alexa – 5M
http://www.alexa.com/siteinfo/nameofsite.com
▪ Make a new spreadsheet with companies as column A and column B
labeled “time spent”/”minutes on site”
▪ Enter data from Alexa in column B.
▪ Mark date range, source, URL for your data (click ? - trailing 3 mo)
OxfordTech.us | GA: Web Metrics l August 2014
175. A. Time Spent- Graph
▪ Select all and sort by column B
▪ Select data, then choose Insert > Column Chart while Data is
selected.
OxfordTech.us | GA: Web Metrics l August 2014
177. B. Bounce Rate- Alexa
http://www.alexa.com/siteinfo/nameofsite.com
▪ Make a new chart tab with same companies as column A and
column B “Bounce” - under a new tab (copy chart, delete rows 2 on)
▪ Enter data on % Bounce in column B as a whole # (“78” for 78%)
▪ Mark date range, source, URL for your data (click ? - trailing 3 mo)
OxfordTech.us | GA: Web Metrics l August 2014
178. B. Bounce Rate- Graph & Analyze
▪ Select all and sort by column B
▪ Select data, then choose Insert > Column Chart while Data is
selected.
▪ Is this a problem?
▪ ASK WHY??
OxfordTech.us | GA: Web Metrics l August 2014
179. C. Traffic Sources- SimilarWeb – 5M
http://www.similarweb.com/website/nameofsite.com
▪ Make a new chart tab with same companies as column A and
column B “Social” - under a new tab (copy chart, delete rows 2 on)
▪ Enter data on % Social in column B as a whole # (“78” for 78%)
▪ (Add other columns for other traffic types. Label each column)
!
!
OxfordTech.us | GA: Web Metrics l August 2014
180. C. Traffic Sources-% Social
▪ Select all and sort by column B
▪ Select all data, then choose Insert > Column Chart while Data is
selected.
!
!
OxfordTech.us | GA: Web Metrics l August 2014
181. D. Demographics- Quantcast – 5M
https://www.quantcast.com/nameofsite.com
▪ Use Quantcast on quantified sites. Mark date range, source, URL
▪ Examples: A&E (Aetv.com - partial), Bleacher Report, TMZ, Gawker,
The Onion, Quantcast.com
▪ Gender tab (not gender “Index”)
▪ Make a new chart with same companies as first tab- under a new tab
(copy chart, delete rows 2 on)
▪ Enter data on % Male in column B as a whole # (“78” for 78%)
▪ Do the same for Female in column C
!
!
Site X
OxfordTech.us | GA: Web Metrics l August 2014
182. D. Quantcast - when there’s Data
▪ Go to https://www.quantcast.com/nameofsite.com
OxfordTech.us | GA: Web Metrics l August 2014
183. D. Demographics- Gender- Alexa
▪ Pro: Available free. Con: Relative Rank #s
▪ Can get real #s for $50/mo
▪ If don’t want to pay and just want to get comparisons- can assign
numeric estimates to Alexa bars, assuming linear scale (-5 to +5)
!
OxfordTech.us | GA: Web Metrics l August 2014
184. D. Demographics- Gender
▪ Select all and sort by column B
▪ Select all data, then choose Insert > Column Chart while Data is
selected.
OxfordTech.us | GA: Web Metrics l August 2014
185. D. Demographics- Gender -Alt View
Always consider the best visualization of data
OxfordTech.us | GA: Web Metrics l August 2014
186. D. Demographics- Gender (Single Site)
Always consider the best visualization of data
OxfordTech.us | GA: Web Metrics l August 2014
187. E. Uniques over Time- Compete – 5 M
▪ https://siteanalytics.compete.com/nameofsite.com
▪ 6 Month Tab. Mouseover Dots.
▪ Make a new tab with same companies as column A and columns B-G each a
month - under a new tab (copy chart, delete rows B on)
▪ Manually Enter Uniques Data into columns B-G
▪ Column H: mark a column for Average
OxfordTech.us | GA: Web Metrics l August 2014
188. E. Uniques over Time Spreadsheet
▪ First Column: Sites. Columns B-G: 6 Months data as #s with labels
▪ Mark the date range, source, URL for your data
▪ Add an AVERAGES column on right. Leave blank for now
OxfordTech.us | GA: Web Metrics l August 2014
189. E. Uniques over Time- Graph! – 5 M
▪ For 5 Sites & 6 Months- Select cells 1A-6G (include months, names of
sites and numbers, but excluding average column)
▪ Choose Insert > Line Chart while Data is selected.
▪ Right-Click on the new chart. Choose “Select Data”
▪ Click “Switch Row/ Column”, then OK (“Plot Rows as Series” on Mac)
OxfordTech.us | GA: Web Metrics l August 2014
190. E. Uniques- Outliers
▪ Sometimes Excluding Data in Graph Can Reveal More Detail
OxfordTech.us | GA: Web Metrics l August 2014
191. E. Bounce Rate alternate- Similarweb -5M
▪ http://www.similarweb.com/website/nameofsite.com
▪ Open up original Bounce Rate Tab
▪ Use to compare against Alexa- find anomalies
▪ Add 1 more Column- Enter Alexa Data (and note source!)
▪ Sort by both columns
OxfordTech.us | GA: Web Metrics l August 2014
192. E. Bounce- compare – 5 M
▪ Eyeball for major differences in Data
▪ Do a double graph of both values (Insert > Chart - bar chart)
▪ Anything interesting?
OxfordTech.us | GA: Web Metrics l August 2014
193. F. Your KPIs - Download, Graph in a new tab
▪ Download/ manually transcribe some other interesting data points
▪ Ex: Bounce Rate, Page Views, Visits/Unique
▪ ALEXA: http://www.alexa.com/siteinfo/nameofsite.com
▪ COMPETE: https://siteanalytics.compete.com/nameofsite.com
▪ QUANTCAST: https://www.quantcast.com/nameofsite.com
▪ SIMILARWEB: http://www.similarweb.com/website/nameofsite.com
OxfordTech.us | GA: Web Metrics l August 2014
195. Review, Analyze & Think!
▪ DON’T BE A ROBOT.
▪ Does the data make sense?
▪ Do the metrics match up across sources?
▪ Double check anything suspicious.
▪ Note anomalies
http://m.cdn.blog.hu/in/investo/image/Robot.jpg
OxfordTech.us | GA: Web Metrics l August 2014
196. Unlock the SEXY. Write relevance for UX
OxfordTech.us | GA: Web Metrics l August 2014
197. Scatterplot- Find Patterns – 10 M
▪ Use your data for % social vs. Age or any other point to form a scatter
▪ Make a new tab, copying over all of the % Social Data
▪ Sort by Site Name (column A)
▪ Go to Age tab, sort by Site Name (col A), then copy Column B data
▪ Go back to new Scatter tab, paste in Age data in Column C
▪ Make sure both tabs were sorted by name first to match the data
OxfordTech.us | GA: Web Metrics l August 2014
198. Scatterplot- Look for Patterns
▪ After sorting, choose 2 columns of just the data- not labels
▪ Sort data by one column (or both)
▪ Insert Scatterplot
▪ May need to invert to plot X:Y instead of Y:X (edit chart data > plot rows/
columns as series in lower left on a Mac; on a PC, invert plot)
▪ Add labels to graph, cross-check graph values
OxfordTech.us | GA: Web Metrics l August 2014
199. Scatterplot- Any Patterns? Make More!
▪ Won’t always get interesting results…. make more!
OxfordTech.us | GA: Web Metrics l August 2014
200. Scatterplot- Correlation vs. Causation
▪ ASK WHY?
▪ Be careful with your conclusions
▪ Regression analysis
▪ Consider alternate reasons
OxfordTech.us | GA: Web Metrics l August 2014
201. Review & Have Details Documented
▪ Document in Excel & Presentations
▪ Sources
▪ Date ranges- exact
▪ Date data was obtained
▪ Name of metric
▪ Source URLs
OxfordTech.us | GA: Web Metrics l August 2014
204. If Time: Examine top Performers & ask…
HOW?
OxfordTech.us | GA: Web Metrics l August 2014
http://openclipart.org/image/2400px/svg_to_png/185271/ico_light_bulb_2.png
205. Asking How- Documenting
▪ Work in Teams
▪ Define “success metrics” & KPIs
before examining
!
▪ Take Screenshots
▪ Make notes on pages & note URLs
▪ See what’s similar/ different
across sites
▪ See what stands out
▪ Form hypotheses on how/ why
OxfordTech.us | GA: Web Metrics l August 2014
206. “Quantifying” your Approach
▪ Define “success metrics” & KPIs and features/approaches that support
them
▪ Rank competitors based on those features/ approaches
▪ Graph your results
OxfordTech.us | GA: Web Metrics l August 2014
215. UX + WEB METRICS
How to Benchmark, Measure
& Evaluate UX Impact
General Assembly
August 03, 2014
THANKS!
!
!
Bev May
Beverly@OxfordTech.us @OxfordTech OxfordTech.us
@UXAwards UXAwards.org