SPSNYC15 - An Independent Evaluation of Third-Party SharePoint Analytics Offerings

SharePoint Saturday New York City 2015
#SPSNYC
Fotopedia.com
An Independent Evaluation of Third-Party SharePoint Analytics Offerings
Jonathan Ralton
All trademarks and registered trademarks are
the property of their respective owners.

Jonathan Ralton
BlueMetal
An Independent Evaluation of Third-
Party SharePoint Analytics Offerings

Agenda
Setting the Stage
Client Story
Measuring and Trending
Definitions
Framework
Examples
SharePoint Analytics
Third-Party Tool Candidates
Which would you choose?
Epilogue
Which did I choose?
How does the story end?
Wrapping Up
Questions

Setting the Stage

Audience
What roles are
you in?
What SharePoint phase
are you in?
What are you hoping
to learn?

Presenter
Jonathan Ralton
• Senior Information Architect
• SharePoint IT Pro since 2005
(WSS/SPS)
• No coding!
• Document Management,
Content Management,
Knowledge Management…
@jonralton
jonathanr@bluemetal.com
blog.jonralton.net

Client Story

Overview
• Food and Beverage Manufacturing Company
• One of the top industry leaders in the country
• One of the top industry leaders in the world
• Employees in dozens of countries
• Hundreds of household brand names
• Some with $1b market shares

Intranet
• Redesigned by BlueMetal and relaunched in late 2014
• Heavy on video content
• Social focus
• Personalization

Challenges
Google
External
Media
Outlets
Chatter/
Rumor
Mill
Emails
Bing

Challenges
Communication Justification
• Need to prepare for new
management oversight
• Need to publish appropriate
news at appropriate times
• Want to maximize investment in
the new facility of the intranet

Objectives
Content and Visitors
• Learn what content is being
consumed and by whom
• Learn which types of content
are most engaging
• Gain better insight into what
kinds of content employees will
respond well to, and in turn
continue to return to
myCorptown for
Employee Communications
• Help to justify team’s value in
managing myCorptown
• Demonstrate higher employee
satisfaction in being informed
• Demonstrate higher employee
engagement

Objectives
Adoption
Engagement

Visitors/Visits/Constituencies
New
Employees
New Users Lapsed Users
Executive
Employees
Mobile Users
Tablet Users Laptop Users
Desktop
Users
Kiosk Users
Hourly
Employees
Salaried
Employees
Departments Off-Hours Work Hours Break Time

Measuring and Trending
Must Love Term Sets

WEB ANALYTICS
def·i·ni·tion [dèffə nísh'n]

Web Analytics
Web analytics is the
measurement, collection, analysis, and reporting
of web data for purposes of
understanding and optimizing web usage.

METRIC

Metric
A metric is a something that can be
measured based on quantifiable data,
or at the very least,
estimated based on anecdotal data.

Metrics w/Tools
Web analytics tools mainly provide metrics for
consumption via reports and APIs.

KEY PERFORMANCE INDICATOR
(KPI)

Key Performance Indicator
A key performance indicator (KPI) is
a visual indicator that tells a story
about one or more measurements, in context.

Key Performance Indicators w/Tools
Web analytics tools may already provide
some KPIs in dashboards.

Metrics and KPI Framework
Goals Signals Metrics KPIs

Goals
Adoption
Engagement
RetentionTask Success
Happiness

Sample Goals – Adoption
Ensure new employees and
new users are engaging
with myCorptown content
for the first time, including
the email newsletter
Build employee buy-in by
telling our story through
our vehicles
Reach our employees
‘where they are’ including
via mobile and kiosks
‘Give them what they want’
by being able to respond
to popularity of content
Get our employees to think
of going to myCorptown
first

Sample Goals – Engagement
Get our employees to
give feedback on content
including likes and
comments, submitting
questions, etc.
Have our employees be
conversant in an
informed manner about
company news with
fellow colleagues
Get our executive team
to choose myCorptown
as their communication
vehicle
Get our employees to
help each other
(Employees as
Communicators)

Sample Goals – Retention
Get our users to return
to myCorptown more
often (more than 1x per
week)
Encourage our lapsed
users to return to
myCorptown
Reduce stale content in
areas on myCorptown
where dynamic content
is expected by our users

Sample Goals – Task Success
‘Give them what they
need’ in addition to
what our users want,
including via search
Solve our users’
problems via self-service
where possible (without
a help desk call)

Sample Goals – Happiness
Ensure our employees are able
to articulate the company
story
• What we stand for
• How does my work fit into the story
• How does my work touch consumers
Support a more transparent
culture at the company
Decrease frustration during
activities on myCorptown
Support our employees
through times of change

Sample Signals
Goal
Get our users to return to
myCorptown more often (more
than 1x per week)
Signals
• Providing content relevant to
users
• Identify users who return to visit
the site within a week period

Sample Signals
Goal
Get our employees to give
feedback on content including
likes and comments, submitting
questions, etc.
Signals
• Identifying posts that have
higher numbers of views
• Identifying posts that have high
numbers of comments
• Monitoring submitted questions

Sample Signals
Goal
‘Give them what they need’ in
addition to what our users want,
including via search
Signals
• Identifying usage of tools and
links that have typically been
difficult to find
• Monitoring search results for
successful and failed queries

Sample KPI w/Trending #1
• Number of retained user home
page views minus bounces as
percentage of all views
• Over time period of 7 days
• Indication of current metric
against target w/in a certain
window
• Trend
57%
Stickiness
Home Page
Target
70%
Window
7 Days

Sample KPI w/Trending #2
• Number of views during off-
work hour segments
window
• Trend
1.6k
Time-Based Visits
Off-Work
Target
2k
Window
5 Days

Sample KPI w/Trending and History #1
• Number of posts related to
influencers
window
• Performance against last
window and previous year’s
window
• Example of where higher metric
is better
38
Influencers
Posts
Target
50
Window
30 Days
YoY
+70.1%
Last Month
-23.9%

Sample KPI w/Trending and History #2
• Number of questions submitted
to portal team
window
window
• Example of where lower metric
is better
12
Portal
Questions Submitted
Target
15
Window
30 Days
YoY
-156.5%
Last Month
-10.4%

Sample KPI w/Snapshot #1
Average Length of Visit
> 5 m > 4 m > 3 m > 2 m > 1 m
• Average visit length
• Segmented into groups
• Current metric for one window
• Shows spread/breakdown detail
• Does not show trend

• Average visit length in minutes
window
• Trend
1.8
Length of Visit
Home Page
Target
2.0
Window
7 Days

• Rating of frustration level on
portal from survey
• Current metric for one window
• Shows spread/breakdown detail
• Does not show trend
0
1000
2000
3000
4000
5000
6000
Low Medium High
Frustration Level
Frustration Level

• Most common rating of
frustration level from survey
• Over time period of 1 fiscal
quarter
window
window
• Example of where metric is
abstracted
M
Portal
Frustration Level
Target
M
Window
Quarter
YoY
+27.6%
Last Quarter
+5.8%

Changes ⓮ → ⓯
• Web Analytics Web Part is not supported anymore
• Reports Removed
• Browser Traffic
• Top Users
• Referring URL

Changes ⓮ → ⓯
• Engine finds relevant information based on clicks, views, etc.
• There are now visual indicators and usage numbers based on number
of views and number of unique visitors
• Clicks and views are counted for each document
• Content can be recommended
• Search results influenced by the priority of an item
• Ability to sort search results by "hit"
• Engine is now extensible for third-party solutions

SharePoint Usage Reporting
Usage
This report shows historical usage
information about the site collection/site,
such as the number of views and unique
users. Use this report to identify usage
trends and to determine times of high and
low activity.

SharePoint Search Reporting
Number of Queries
This report shows the number of search
queries performed. Use this report to
identify search query volume trends and to
determine times of high and low search
activity.
Top Queries by Day/Month
This report shows the most popular search
queries. Use this report to understand what
types of information visitors are seeking.
Abandoned Queries by Day/Month
This report shows popular search queries
that received low click-through. Use this
report to identify search queries that might
create user dissatisfaction and to improve
the discoverability of content. Then, consider
using query rules to improve the query's
results.
No Result Queries by Day/Month
This report shows popular search queries
that returned no results. Use this report to
identify search queries that might create
user dissatisfaction and to improve the
discoverability of content. Then, consider
using query rules to improve the query's
results.
Query Rule Usage by Day/Month
This report shows how often query rules
trigger, how many dictionary terms they use,
and how often users click their promoted
results. Use this report to see how useful
your query rules and promoted results are to
users.

SharePoint Library Reporting
Most Views
Most Views by Unique Users
Most Recommendation Clicks

Third-Party Tool Candidates

Requirements – Compatibility
A tool for myCorptown
analytics will require
compatibility with (or
extensibility to support)
traffic delivered through…

Requirements – Awareness
In order to provide the most
visibility into activity on
myCorptown, a tool for
analytics should be
specifically aware of traffic,
content, and actions related
to… • Social Intranet
• Video Cloud

Requirements – Content Segmentation
Intranet administrators would
like to segment analytics data
by the types of content
consumed.
This would require support
for…
• Metadata
• File Extension

Requirements – Visitor Segmentation
Along with the desire to
segment the content
consumed within
myCorptown, intranet
administrators will need to
examine various properties of
the visits and visitor as well.
These attributes of a visit or
visitor should be supported…
• Platform
• Mobile Device
• Browser
• Date and Time
• Session
• Microsoft SharePoint 2013
• User Profile Property
• User Audience Membership
• Microsoft Active Directory
• User
• User Attribute
• User Group Membership

Requirements – Extensibility
Reusable Metrics
In order for a dashboard or other
display and recalculation of raw
metric data into KPIs and reports,
an API or REST interface should
ideally be provided by the tool.
Email Campaigns
In addition to native web traffic
on myCorptown, the tool should
support UTM tags commonly
used in conjunction with email
campaigns and the hyperlinks
that refer visitors to the site.

Requirements – Pricing
For the purposes of
evaluation, the following
data points were used:
• Page Views
~1,000,000/month
• Users
~30,000

Google Analytics
Google Analytics helps analyze visitor
traffic and paint a picture of your
audience and their needs. Mainly
designed for e-commerce websites, it
is a cloud-only/SaaS offering.

Google Analytics
Trackable Attributes/Actions
• Page Views
• Clickpaths
• Visitors
• Browser/Platform/Device
• Date and Time
Analysis Method
• Tracking occurs via JavaScript-
based page tags

Google Analytics
• Data is often based on a subset, or percentage of sessions
• This is due to the practice of reporting trends in the subset to reduce
processing time
• This can occur in the reports, during the data collection itself, or in both
places

Google Analytics
Key Benefits Key Drawbacks
 In-Page Analytics
 Clickpath Visualization
 Sparkline KPIs
 Real-time
 Cloud-based
 Google takes ownership of your metrics
data and may use it how they see fit
 Not ‘SharePoint-aware’
 Does not analyze search queries
 No video metrics
 No Yammer integration
 No Sitrion integration
 Visitor attributes require customization
 Shared login/Google account required

ActualMetrics
Angelfish
With its roots in Urchin, Angelfish
analyzes IIS logs as a completely
separate application, usually installed
on a server apart from SharePoint.

ActualMetrics Angelfish
• Page Views
• Clickpaths
• File Downloads
• Visitors
• Date and Time
Analysis Method
• Angelfish relies on the IIS logs
• Although tagging with JavaScript
is supported, it is not required

• This tool appears to be in its infancy at this point in time
• A lot of features are in the works/slated for a future release
• The audience for this tool seems to be more technical users than
business users
• The interface seems like it would be more difficult to navigate for someone
outside of IT

 Spike analysis—second-level time
segmentation
 Unlimited nested segments
 Bandwidth analysis
 Broken link analysis
 Six different tracking methods for visitors
 Reports are only exportable in unformatted
form (CSV, XML)
 Separate hardware recommended

Adobe Analytics
Formerly known under the brand
Omniture, Adobe purchased this
analytics suite and it is now part of
their Marketing Cloud offering. Mainly
designed for e-commerce websites, it
is a cloud-only/SaaS offering.

Adobe Analytics
• Page Views
• Clickpaths
• Visitors
• Date and Time
Analysis Method
• Tracking occurs real-time via
JavaScript-based page tags

Adobe Analytics
• Adobe has put a lot of effort into the user interface to enable
visualization of data

Adobe Analytics
 Sparkline KPIs
 Real-time
 Cloud-based; low overhead

HarePoint Analytics
for SharePoint
Using its own analytics engine
separate from SharePoint and any
server logs, HarePoint Analytics installs
as part of SharePoint while using its
own SQL databases.

HarePoint Analytics for SharePoint
• Page Views
• Clickpaths
• File Downloads
• Search Queries
• Visitors
• Date and Time
Analysis Method
• Not requiring JavaScript tags,
the tool utilizes its own tracking
within SharePoint to provide
metrics

• HarePoint seems committed to this product, as it has adapted it from
SharePoint 2007 through 2010 and 2013, however they have yet to
integrate social metrics

 Workflows tracked
 Web part for analytics display within
SharePoint
 Aware of content within SharePoint, not
just actions by visitors
o Unused content can be identified
 Real-time
 No additional hardware required
 Must access reports from Central
Administration
 Can’t track external links
 No social tracking

Intlock CardioLog
Analytics
Designed specifically for SharePoint
and covers all aspects of usage
including portals, extranets,
collaboration sites, and social. It is
installed on a separate SQL server.

Intlock CardioLog Analytics
• Page Views
• Clickpaths
• File Downloads
• Search Queries
• Visitors
• Date and Time
Analysis Method
• Tracking occurs real-time via
JavaScript-based page tags
• IIS logs can be imported to
integrate historic data
• It does not rely on SharePoint’s
built-in analytics or logs

Intlock CardioLog Analytics
• This tool has great in-page feedback options including:
• Message bar
• Survey
• A/B multivariate testing

Cardio
 Custom dashboards
 Direct integration with Active Directory and
SharePoint user profiles
 Display within SharePoint
 Page metadata
 Automatically generated reports
 Identify unused content
 Sitrion integration
 Requires separate hardware

Piwik SharePoint
Analytics
With a highly evolved UI for reports,
Piwik offers solid SharePoint analytics
by activating a feature per site
collection.

Piwik SharePoint Analytics
• Page Views
• Clickpaths
• File Downloads
• Search Queries
• Visitors
• Date and Time
Analysis Method
• Tracking occurs via JavaScript-
based page tags

• The installation of Piwik is very streamlined
• A pre-configured server image is delivered and the tool automatically
populates the tracking code into all relevant areas

 Real-time viewing of visitor actions
 Sparkline KPIs
 Ease of setup
 Lack of social tracking
 Separate hardware required

WebTrends
Closely aligned with Microsoft,
Webtrends is a marquee vendor for
web analytics.

• Page Views
• Clickpaths
• File Downloads
• Search Queries
• Visitors
• Date and Time
Analysis Method
• Embedded JavaScript tags in the
page send information
(encrypted in transit) to the tool

Webtrends SharePoint Analytics
• Webtrends seems to have a particular focus on adoption and
engagement.
• Their pitch is “consumable and actionable insights for business users”
• It has a well-polished UI

 Cloud-based; no overhead
 Targeted for consumption by business
users
 Plug-ins for Brightcove
 Sitrion integration
 Feature-heavy product; only small subset of
feature set is required

Which would you choose?

Epilogue

Areas of Comparison
• Compatibility
• Tracking Method
• On-Premise vs. SaaS
• Content and Process Awareness
• Content Segmentation
• Visits/Visitor Segmentation
• Reporting
• Feedback
• Extensibility
• Price

Which did I choose?

Why?
• This recommendation is based largely on the feature set offered by
CardioLog
• How those features align with the needs of the client in tracking activity on
myCorptown
• Content as well as activity is monitored
• Lack of action can be tracked on documents and items
• In-page feedback mechanisms were very attractive

Why?
• Sitrion analytics are built-in and
will require little to no
customization
• Brightcove analytics should be
able to be integrated via their
API and brought into CardioLog

Why?
• Other tools…
• Did not have as extensive a feature set to support the client’s requirements
• Were less SharePoint-specific
• Did not allow for business users to consume their reports as easily
• Cost too much

How does the story end?

…and they lived happily ever after.
• Decided to address some of
their near-term goals
• Have reports against them based
on certain metrics
• Some of the KPIs would be able to
be developed from those metrics
(manually)
• In order for this to take
place, an augmentation of
current information being
sent over the Internet to
Google Analytics is required
• This needs to include sending
over key user attributes as well as
some classification of the pages
being viewed
• Integration with SharePoint user
profile data will be required to
track things such as a user’s
department

Warnings
• Complete user information such as first and last names should not be
sent over to Google Analytics, as the data may be reused by Google
in any manner
• Steps should be taken so that any confidential information or trade
secrets are not accidentally injected into the Google ecosystem in the
form of metatags, page titles, etc.

Wrapping Up

Wrapping Up
The views expressed in this
presentation are not in any way
endorsed by any of the product
vendors, and I warrant no
accuracy in the data presented.

Wrapping Up
Although I concluded that
CardioLog Analytics is the leading
candidate for this particular
engagement, this does not by any
means signify future
disqualification by myself or
BlueMetal in future evaluations of
the other tool candidates.

Wrapping Up
In fact, another tool might be the
best we could recommend to a
completely different future client
depending on their needs.

Questions

THANK YOU
EVENT SPONSORS
We appreciate you supporting the
New York SharePoint Community!
• Diamond, Platinum, Gold, and
Silver have tables scattered
throughout
• Please visit them and inquire about
their products & services
• To be eligible for prizes make sure to
get your bingo card stamped by ALL
sponsors

We Are Hiring
in NYC!
• Mobile Web Software Engineer
• Cloud & Services Senior Architect
• Senior Java Developer
• Senior Web Developer
• Senior SharePoint Engineer
• Devices & Mobility Associate Software
Engineer
• Operations Coordinator (Part Time)
Learn more:
https://www.linkedin.com/company/bluemetal-architects/careers

Reference

Resources
Tool Website Tool Download/Request Tool Demo
ActualMetrics Angelfish Link
Adobe Analytics Managed Metadata Overview Link
Google Analytics
HarePoint Analytics for SharePoint Link Link
Intlock CardioLog Analytics Link

Google Analytics for SharePoint 2013/Office 365 (CodePlex)
Resources

SPSNYC15 - An Independent Evaluation of Third-Party SharePoint Analytics Offerings

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a SPSNYC15 - An Independent Evaluation of Third-Party SharePoint Analytics Offerings

Similar a SPSNYC15 - An Independent Evaluation of Third-Party SharePoint Analytics Offerings (20)

Más de Jonathan Ralton

Más de Jonathan Ralton (20)

Último

Último (20)

SPSNYC15 - An Independent Evaluation of Third-Party SharePoint Analytics Offerings

Notas del editor