As open access journals and repositories gain a foothold in scholarly communication, researchers are finding that the traditional impact factor and citation count metrics only reflect a portion of the dissemination of scholarly works. New technology, research, and citation tools aid our ability to measure the influence of research. A matrix of tools and initiatives, like PLoS Article-Level Metrics, BePress’ Author Dashboard, Mendeley, Altmetrics, and ImpactStory are providing a more robust picture of scholarly communication today. This presentation provides an overview of the impact factor system and new tools for gathering metrics and their relevance for librarians and researchers. Presentation given at the Library Information Technology Association (LITA) Forum in Louisville, KY, in November 2013.

1. BRACE FOR IMPACT: NEW
MEANS OF MEASURING
RESEARCH METRICS
MARY ELLEN SLOANE,
SCHOLARLY COMMUNICATIONS LIBRARIAN
FOR BASIC AND APPLIED SCIENCES,
MIDDLE TENNESSEE STATE UNIVERSITY

2. OUTLINE
• Impact Factor system
• Evolution of Scholarly Communication
• Altmetrics sources
• Open access platforms
• Sharing platforms
• Saving platforms
• Hacking altmetrics
• Impact of altmetrics

3. SUMMARY
• As open access journals and repositories gain a foothold
in scholarly communication, researchers are finding that
the traditional impact factor and citation count metrics
only reflect a portion of the dissemination of scholarly
works.
• New technology, research, and citation tools aid our
ability to measure the influence of research. A matrix of
tools and initiatives, like PLoS Article-Level Metrics,
BePress’ Author Dashboard, Mendeley, Altmetrics, and
ImpactStory are providing a more robust picture of
scholarly communication today.
• This presentation will provide an overview of the impact
factor system and new tools for gathering metrics and
their relevance for librarians and researchers.

4. IMPACT FACTOR SYSTEM
• The impact factor system measures the number of
citations for a given journal over a period of time.

5. IMPACT FACTOR SYSTEM

6. IMPACT FACTOR SYSTEM
•
•
•
•
•
•
•
•
•
Not always a good indicator of quality research
Journal ‘impact’ is relative to an article’s value
Not necessarily meaningful to authors or researchers
Self-citations can skew results
Focuses on overall impact, as opposed to articlelevel measures
Review journals have more citations
San Francisco Declaration on Research Assessment –
concerns with impact system
Open access sources may not be represented
Tenure and promotion criteria tied to publishing

7. EIGENFACTOR

8. EVOLUTION OF SCHOLARLY
COMMUNICATION
New means of publishing
Institutional repositories
Sharing data sets
Sharing code
Sharing presentations
Sharing video
Article-level metrics can help encourage publishing
in open-access journals and repositories
• Open access publishing and sharing is increasing
• Social networks can evolve organically and include
open access and other sources of sharing
•
•
•
•
•
•
•

9. EVOLUTION OF SCHOLARLY
COMMUNICATION
• Ability to measure use by readers, researchers
• Earlier indicators of research impact
• Web of Science:
• Mendeley:

10. EVOLUTION OF SCHOLARLY
COMMUNICATION

11. METRICS SOURCES
•
•
•
•
•
•
•
•
•
Social media
Blog posts, Wikipedia
Sharing links to publisher full text
Open access views, downloads, citations, followers
Bookmarks, favorites
Citation saving and groups
Worldcat holdings
Standard citation counts
PlumX - http://www.plumanalytics.com/metrics.html

12. METRICS SOURCES
• Open access sources:
•
•
•
•
•
•
Dryad
Figshare
GitHub
BePress
dSpace
PLOS
• Saves
• Mendeley,
• citeulike
• Delicious
• Uses
• Youtube, Vimeo
• Slideshare
• Shares
• Twitter
• Facebook
• Bit.ly
• Citations
•
•
•
•
Web of Science
Scopus
Google Scholar
PubMed
• Aggregate metrics
• Altmetrics, and
• ImpactStory
• Plum X

13. WEB OF SCIENCE ARTICLE METRICS
• C&RL Impact Factor:
• Citation map:

14. OPEN ACCESS - PLOS

15. OPEN ACCESS - FIGSHARE

16. OPEN ACCESS - DRYAD

17. INSTITUTIONAL REPOSITORIES

18. IF YOU CAN’T BEAT THEM, JOIN THEM
• Articles Metrics from publishers

19. ACCURACY PROBLEMS

20. IMPACT STORY

21. ALTMETRIC

22. PLUMX
• PlumX Tracks Groups
•
•
•
•
•
•
•
•
•
•
•
University
Institute
Museum
Department
Lab
Company
Publisher
Journal
Volume
Issue
Collection

23. SO WHAT
?
• How many are using social media and
reference managers in meaningful ways?
• How meaningful are the metrics?
• Links between related items, article and data
set
• Can easily be gamed
• Debate over early downloads as indicator of
later citation
• Limited to DOI in some cases
• Disciplines may use different platforms and tools

24. HACKING METRICS
• Does this milk smell bad?

25. IMPACT OF ALTMETRICS
•
•
•
•
•
•
•
•
New ways to perform bibliometrics
New ways to do research
New ways to share and connect
Recent articles will have more altmetrics
Discipline-specific measures of impact
Tenure and promotion considerations
Researcher’s dissemination will evolve
At some point will become customary

26. COMING SOON
• NISO Alternative Assessment Metrics (Altmetrics)
Project
• Here Comes the Sunburst: Measuring and
Visualizing Scholarly Impact – Sunday 11/10 – 9:00
am