About the Webinar In a time of shrinking budgets and growing reliance on electronic resources, the collection and analysis of usage statistics has become a staple of the library world. But while usage statistics may be ubiquitous, many librarians still struggle with the best methods of interpreting the data. The ability to effectively understand and apply usage data is an important skill for librarians to master as they attempt to analyze their collections and justify their expenses to administrations. This webinar will highlight the ins and outs of COUNTER, as well as discuss the process of analyzing the data once harvested.Introductions Agenda Todd Carpenter, Executive Director, NISO Todd Enoch, Head, Serials and Electronic Resources, University of North Texas Libraries; Chair of the Continuing Education Committee, NASIG * * * * * * * COUNTER Update: Release 4 of the COUNTER Code of Practice for e-Resources Peter Shepherd, Project Director, COUNTER Integrating COUNTER Statistics within the Information Workflow Oliver Pesch, Chief Product Strategist and Senior Vice President, EBSCO Information Services Usage in the Eye of the Beholder: Developing Academic Library Usage Reports that Meet the Needs of Your Institution Jill Emery, Collection Development Librarian, Portland State University Library

1. NISO/NASIG Joint Webinar:
Playing the Numbers:
Best Practices in Acquiring, Interpreting and
Applying Usage Statistics
May 21, 2014
Speakers:
Peter Shepherd, Project Director,
COUNTER
Oliver Pesch, Chief Product Strategist and Senior Vice President,
EBSCO Information Services
Jill Emery, Collection Development Librarian,
Portland State University Library
http://www.niso.org/news/events/2014/nasig/

2. Release 4 of the COUNTER
Code of Practice for e-
Resources and new usage-
based measures of impact
Peter Shepherd
COUNTER
May 2014

3. COUNTER Release 4
- objectives
 A single, unified Code covering all e-resources,
including journals, databases, books, reference works,
multimedia content, etc.
 Improve the database reports
 Improve the reporting of archive usage
 Enable the reporting of mobile usage separately
 Expand the categories of ‘Access Denied’ covered
 Improve the application of XML and SUSHI in the
design of the usage reports
 Collect metadata that facilitates the linking of usage
statistics to other datasets, such as subscription
information

4. Release 4: main features
 A single, integrated Code of Practice covering
journals, databases, books, reference works
and multimedia content
 An expanded list of Definitions, including terms
such as ‘Gold Open Access’, ‘Multimedia Full
Content Unit’, ‘Record View’, ‘Result Click’, as
well as different categories of ‘Access Denied’,
etc. that are used for the first time in Release 4
 Enhancements of the SUSHI (Standardised
Usage Statistics Harvesting Initiative) protocol
designed to facilitate its implementation by
vendors and its use by librarians

5. Release 4: main features
 A requirement that Institutional Identifiers, Journal DOI
and Book DOI be included in the usage reports, to
facilitate not only the management of usage data, but
also the linking of usage data to other data relevant to
collections of online content.
 A requirement that usage of Gold Open Access articles
within journals be reported separately in a new report:
Journal Report 1 GOA: Number of Successful Gold Open
Access Full-text Article Requests by Month and Journal.
 A requirement that Journal Report 5 must be provided

6. Release 4: main features
 Modified Database Reports, in which the previous requirement to report
Session counts has been dropped, and new requirements, to report
Record Views and Result Clicks, have been added. (Database Report 3
has also been renamed Platform Report 1).
 A new report, Multimedia Report 1, which covers the usage of non-
textual multimedia resources, such as audio, video and images, by
reporting the number of successful requests for multimedia full content
units
 New optional reports covering usage on mobile devices
 A description of the relative advantages of logfiles and page tags as the
basis for counting online usage
 Flexibility in the usage reporting period that allows customers to specify
a date range for their usage reports

7. Release 4: Standard Usage
Reports
 Journal Report 1: Number of Successful Full-Text Article Requests by Month and
Journal
 Journal Report 1 GOA: Number of Successful Gold Open Access Full-Text Article
Requests by Month and Journal
 Journal Report 2: Access Denied to Full-Text Articles by Month, Journal and
Category
 Journal Report 5: Number of Successful Full-Text Article Requests by Year-of-
Publication (YOP) and Journal
 Database Report 1: Total Searches, Result Clicks and Record Views by Month
and Database
 Database Report 2: Access Denied by Month, Database and Category
 Platform Report 1: Total Searches, Result Clicks and Record Views by Month and
Platform
 Book Report 1: Number of Successful Requests by Month and Title
 Book Report 2: Number of Successful Section Requests by Month and Title
 Book Report 3: Access Denied to Content Items by Month, Title and Category
 Book Report 4: Access Denied to Content Items by Month, Platform and Category
 Book Report 5: Total Searches by Month and Title
 Multimedia Report 1: Number of Successful Full Multimedia Content Units
Requests by Month and Collection

8. Release 4: recording and reporting
usage on mobile devices
 The following optional additional reports enable usage on mobile devices to be
reported separately:
 Journal Report 3 Mobile: Number of Successful Item Requests by Month, Journal
and Page Type for usage on a Mobile Device
 Title Report 1 Mobile: Number of Successful Requests for Journal Full-text Articles
and Book Sections by Month and Title ( formatted for normal browsers/delivered to
mobile devices AND formatted for mobile devices/delivered to mobile devices)
 Title Report 3 Mobile: Number of Successful Requests by Month, Title and Page
Type (formatted for normal browsers/delivered to mobile devices AND formatted
for mobile devices/delivered to mobile devices)
COUNTER will recognize as usage on a mobile device, which may be reported in the
above reports, any usage that meets one of the following criteria:
 useragents that are included in the WURFL list. WURFL is the Wireless
Universal Resource FiLe, a database containing the profile of mobile devices; this
database may be found at: http://wurfl.sourceforge.net/
 usage via a proprietary mobile App provided by the publisher/content provider

9. Release 4: timetable for
implementation
 Deadline date for implementation of Release 4:
31 December 2013
-after this date only vendors compliant with Release
4 are COUNTER compliant
 Over 100 Publishers/Vendors are now providing Release
4 Usage Reports

10. Full details of Release 4 will be found on the
COUNTER website at:
http://www.projectcounter.org/code_practice.html
COUNTER Code of Practice
-Release 4

11. New COUNTER usage-based
measures of impact
Advantages:
 Usage can be reported at the individual item and individual researcher level
 Usage is more ’immediate’ than citations
 Usage potentially covers all categories of online publication
 COUNTER usage statistics are independently audited and generally trusted
Two new COUNTER Codes of Practice have been launched:
COUNTER Code of Practice for Articles (COUNTER Articles)
 Recording, consolidation and reporting of usage at the individual article level
 Standard applies to publishers, aggregators and repositories
COUNTER Code of Practice for Usage Factor
 Usage-based measure of impact of journals, institutions and individual scholars
 The Usage Factor for a Journal is the Median Value in a set of ordered full-text article usage data ( i.e. the
number of successful full text article requests) for a specified Usage Period of articles published in a journal
during a specified Publication Period.
COUNTER Articles and Usage Factor are both based on the recording and
consolidation of COUNTER-compliant usage data at the individual article level

12. COUNTER Code of Practice for
Articles
COUNTER Articles covers the following areas:
 article types to be counted;
 article versions to be counted;
 data elements to be measured;
 definitions of these data elements;
 content and format of usage reports;
 requirements for data collection and data processing;
 requirements for independent audit (under development);
Release 1 of the COUNTER Code of Practice for Articles is available on the
COUNTER website at: http://www.projectcounter.org/counterarticles.html

13. Usage Factor: aims and outcomes
The overall aim of the Usage Factor project was to explore how online journal usage
statistics might form the basis of a new measure of journal impact and quality, the
Usage Factor for journals.
Specific objectives were to answer the following questions:
 Will Usage Factor be a statistically meaningful measure?
 Will Usage Factor be accepted by researchers, publishers, librarians and research
institutions?
 Will Usage Factor be statistically credible and robust?
 Is there an organizational and economic model for its implementation that would
cost-effective and be acceptable to the major stakeholder groups.
Following extensive testing using usage data for over 200 journals from a range of
publishers the main outcome of the project has been the new COUNTER Code of
Practice for Usage Factors. This new Code of Practice uses the article level usage
data collected using the COUNTER Code of Practice for Articles as the basis for
the calculation of the Usage Factor.
The COUNTER Code of Practice for Usage Factors is available on the COUNTER
website at: http://www.projectcounter.org/usage_factor.html

14. Who will benefit from the Usage
Factor?
Four major groups will benefit from the introduction of Usage Factors:
 Authors, especially those in practitioner-oriented fields, where citation-
based measures understate the impact of journals, as well as those in
areas outside the core STM fields of pure research, where coverage of
journals by citation-based measures is weak.
 Publishers, especially those with large numbers of journals outside of the
core STM research areas, where there is no reliable, universal measure of
journal impact, because citation-based measures are either inadequate or
non-existent for these fields
 Librarians, when deciding on new journal acquisitions, have no reliable,
global measures of journal impact for fields outside the core STM
research fields. They would use usage-based measures to help them
prioritise journals to be added to their collections.
 Research Funding Agencies, who are seeking a wider range of credible,
consistent quantitative measures of the value and impact of the outputs of
the research that they fund.

15. Usage Factor: Journals
- the calculation
Publishers will be able to generate Usage Factors using the Code of Practice, but will
have to be independently audited for their Usage Factors to be listed in the Usage
Factor Central Registry. Two categories of Usage Factor may be calculated
 The 24 month Journal Usage Factor 2010/2011: all content
The median number of successful requests during 2010/2011 to content
published in the journal in 2010/2011
 The Journal Usage Factor 2010/2011: full-text articles only
The median number of successful requests during 2010/2011 to full-text
articles published in the journal in 2010/2011
 Note:
1.The article-level data collected in COUNTER Article Report 1 will be used as
the basis for the Usage Factor calculation
2. Usage Factors will be reported annually, for 2010/2011, 2011/2012, etc.

16. COUNTER Articles and Usage
Factor - implementation
 Step 1: implement COUNTER Code of Practice
for Articles
 Step 2: Collect article-level usage data for
2014/2015
 Step 3: Calculate and report Usage Factors
using protocols specified in Code of Practice for
Usage Factors

17. Integrating COUNTER Statistics within the
Information Workflow
Oliver Pesch
Chief Product Strategist
EBSCO Information Services

18. Gathering usage is just the first
step…

19. and usage is just one piece of
the puzzle.

20. Effective collection analysis involves
combining usage with other data for a
more complete picture
and presenting that information in a
meaningful context.

21. Examples of usage in the workflow
(presenting usage in context)

22. Subscription renewals

23. Subscription renewals

24. Subscription renewals
Instant access to cost-
per-use and holdings
data streamlines the
renewal process.

25. Individual Titles

26. Individual Titles
Detailed analysis,
holdings and usage
available for individual
orders.

27. Publisher packages

28. Publisher packages Package-level totals
presented

29. Publisher packages
Quick charts to see the
“health” of the package.

30. Publisher Packages

31. Publisher Packages
Usage and cost-per-use
in package details
facilitates decisions on
“swaps” and “drops”.

32. Usage vs other metrics

33. Usage: The Key Metric
% of librarians indicating a metric as very important when making
content decisions
Source: EBSCO, 2012; EBSCO Library Collection and Budgeting Trends Survey, 2012

34. We asked librarians how they use
usage statistics:
98% for cancellation and/or renewal
decisions
72% for journal package negotiations
Source: EBSCO, 2012; EBSCO Library Collection and Budgeting Trends Survey,
2012
Why Usage Statistics Are Important

35. Including other metric types

36. Including other metric types
This library has included
Impact Factor and
Eigenfactor scores.

37. Decisions and challenges related to usage
and collection analysis

38. Usage from all platforms vs publisher platform

39. Usage from all platforms vs publisher platform
This reports provides
analysis for the
publisher platform and
all platforms.
Is this an important title
for this library’s users?

40. Subscriptions to “Combinations”
We have a single order
that is for two titles.

41. Subscriptions to “Combinations”
The order is mapped to
both titles in holdings

42. Subscriptions to “Combinations”
Usage for this “order”
accumulates usage
from both titles so cost-
per-use is meaningful.

43. Mapping the data and analyzing
packages…

44. Putting it all together: the basics
The goal
A report showing subscribed
titles with their cost, use and
cost-per-use.

45. Putting it all together: the basics
Usage Cost

46. Putting it all together: the basics
Usage Cost
How do you
connect usage
to cost? ISSN?

47. Putting it all together: the basics
Usage Cost
How do you
connect usage
to cost? ISSN?
Challenges with ISSN
matching
- Wrong or missing ISSN
- Same ISSN on multiple rows
in COUNTER reports
- Same ISSN on multiple
reports
- Combinations have one cost
item but multiple usage entries
- Titles change publishers
(ISSN changes)
- Publishers change platforms

48. Putting it all together: packages
Usage Cost
The goal
Create a package analysis
with package-level summary.

49. Putting it all together: packages
Usage Cost
How do you
know what is
included in the
package?
Challenges
Analyzing the package as a
unit means accumulating
cost and usage for the entire
package.
COUNTER statistics are
reported at the “Platform”
level so the package content
is not defined in the report.

50. Putting it all together: packages
Usage Cost
How do you
know what is
included in the
package?Solution
Incorporate detailed
holdings and/or package
definition into the analysis.
Holdings

51. Putting it all together: other data
Usage CostHoldings Other
The goal
Add impact factors and other
key-decision data to the
report.

52. Putting it all together: other data
Usage Cost
How do you
connect the other
data? ISSN?
Holdings Other

53. Putting it all together: other data
Usage Cost
How do you
connect the other
data? ISSN?
Holdings Other
Inclusion of impact
factors, faculty
recommendations, do-
not-cancel information
has similar mapping
challenges.

54. Putting it all together
Usage CostHoldings Other
The goal
Make this process
repeatable and as simple as
possible.

55. Putting it all together
Usage CostHoldings Other
What technology can
do all this?

56. Putting it all together
Usage CostHoldings Other
What technology can
do all this?
Some options
• Excel
• MS Access or similar
• Commercial products
with central knowledge
base to handle the
mappings.

57. Summary
• Usage is a means to an end
• Usage needs to be combined with other data to be truly
effective
• Providing usage and related metrics within the workflow
can improve efficiency and quality of decision making
• The act of “combining” usage and other data has several
challenges
• Close sometimes has to be good enough
• Knowledge base centric applications will help with the
mapping

58. Thank you
opesch@ebsco.com

59. Usage in the
Eye of the
Beholder:
Developing Academic
Library Usage Reports
that Meet the Needs of
Your Institution
Jill Emery
Collection Development
Librarian
Portland State University

60. Usage is Contextual

61. Determine the audience
 Different
audiences
require different
data
 Understand
level of detail
sought
LSE, Audience in the Old Theatre, c1981

62. What is to Be Answered?
 Knowing the
outcomes
sought upfront,
saves time with
data gathering
 Review
outcomes with
vested parties

63. Less is More
 Curtail data
 Customize level
of detail
 Be as concise as
possible OSU Special Collections & Archives: Crater
Lake, looking north from the Lodge

64. Gathering Stats
 Sushi
 Set ingest for
right reports
 Next Gen ILS
management

65. JR1 & JR1a
 JR1-standard
 JR1a-optional
 Dedupe between
the two
LSE, Statistics Machine Room, 1964

66. Book Reports
 Growing
demand
 BR1-standard
 BR2-standard

67. Multi-Media Reports
 New with
Counter 4 stats
 Advocate for
providers to
become
compliant
U.S. National Archives: Nitrate Film Stored
in a Brick Vault

68. Gold Open Access
 Another new
report with
Counter 4
 standard report
 Can be used in
price
negotiation

69. Typical Report Requests
 Academic: ACRL,
ARL, State-wide
 Accreditation
 Annual
 Purchasing
 Collection Decision

70. Academic Reports
 High level
 Gross numbers
 Little detail

71. Annual & Accreditation
 High level
 Multi-year
comparisons
 More detail

72. Purchasing Reports
 More granular
detail
 May be multi-
year
 May need to
adjust more
often
Field Museum, Page 74 of Accounting
Administration from Financial Ledger from
World's Columbian Exposition 1893

73. Collection Decision
Reports
 Most detailed
 Look at use and
qualitative
factors
 Multi-year

74. Report Structure
 Remember
audience
 Questions to be
answered
 Level of detail
needed
NASA, Differential Analyzer

75. Graphs/Pie Charts
 Use for annual
reports
 Accreditation
reports
 Purchasing
reports

76. Administrative
 Keep to a page
or short slide
deck (5 slides)
 Use graphs/pie
charts to convey
salient
information
Smithsonian, Mary Blade standing at
the blackboard

77. Purchasing Reports
 Combination of
graphs/charts &
spreadsheets
 Longer to
provide greater
detail

78. Collection Decision
Reports
 Almost always
spreadsheets
 Simplified from
COUNTER
reports
 Contextual
detail California Historical Society: Menu, Ritz Old
Poodle Dog, San Francisco

79. Thank you!
Jill Emery
Collection
Development
Librarian
Portland State
University
jemery@pdx.edu

80. NISO/NASIG Joint Webinar • May 21, 2014
Questions?
All questions will be posted with presenter answers on
the NISO website following the webinar:
http://www.niso.org/news/events/2014/nasig/
NISO/NASIG Webinar:
Playing the Numbers: Best Practices in Acquiring,
Interpreting and Applying Usage Statistics

81. Thank you for joining us today.
Please take a moment to fill out the brief online survey.
We look forward to hearing from you!
THANK YOU