The online reference manager tool Mendeley is one of the most promising tools for altmetrics research (Li, Thelwall and Giustini, 2011;Wouters & Costas, 2012) and it has been already used in other previous studies (Bar-Ilan et al., 2012b; Li, Thelwall and Giustini, 2012; Priem, Piwowar & Hemminger, 2012; Zahedi, Costas & Wouters; 2013). Most of studies investigated how altmetrics capture different type of impact compare to citations (some of them mentioned above); while in others the focus has been on how altmetrics can be used as predictor of citations (Waltman & Costas, 2013); also weak correlation among users’ tag and bookmarks as an indicator of journal usage and perception and citations observed for physical journals (Haustein, & Siebenlist, 2011). In the case of Mendeley, the correlation with citations has been observed to be higher (Bar-Ilan et al., 2012a; Bar-Ilan et al., 2012b; Priem, Piwowar & Hemminger, 2012; Li, Thelwall and Giustini, 2012; Li & Thelwall, 2012; Zahedi, Costas & Wouters; 2013), however, so far the relationship of the different types of readers with the impact of the publications has not yet been explored. For this reason, in this study, we present an exploratory analysis of the patterns of reading of the different types of users in Mendeley and we study their relationship with citations. Thus, our main objective is to know if there are different patterns in terms of impact depending on the different ‘career stages’, ‘disciplines’ and ‘countries’ of the readers in Mendeley. In the case of finding different types of impact and reading patterns among Mendeley readers, this could open the door to detect different types of impact (e.g. education impact or professional impact) and even to introduce the possibility of considering the different users as potential predicting elements of citations. Methodology & preliminary results: In this research we have studied two random samples of publications from the Web of Science: the first one containing 20,000 publications published between 2005 and 2011 from all disciplines, and the second sample include 200,000 publications published between 2011 and 2012 also from all disciplines. Both gathered during March and April 2013 via the Mendeley API and using the DOI of the publications as the linking element. For the two samples we have also calculated standard bibliometric indicators (Waltman et al., 2011). For the analysis of the users we have considered the information of the top three ‘career stage users’, ‘countries’ and ‘disciplines’ of the users. We acknowledge the limitation of counting only with the top three and we discuss this in the paper. Some preliminary results show that PhD students tend to read papers with higher impact than other users and also they read more recent papers. Further research will be done in order to explore other potential factors that can influence this observation.

1. What is the impact of the publications
read by the different Mendeley users?
Could they help to identify alternative
types of impact?
Zohreh Zahedi, Rodrigo Costas & Paul Wouters
Centre for Science and Technology Studies (CWTS)
ALM Workshop, San Francisco, CA, USA
October 10-12, 2013

2. Outline
• Introduction
• Objectives
• Research Questions
• Methodology
• Findings
• Conclusions, Discussions & Limitations
1

3. Introduction
Altmetrics Tools:
Mendeley , Impact Story , altmetric.com,
PLOS ALM, F1000, Plum Analytics, ….
2

4. Previous research
Altmetrics & Citation Correlation:
Henning (2010);Priem, Piwowar & Hemminger (2012);
Bar-Ilan et. al. (2012a & 2012b); Li, Thelwall & Giustini
(2012); Li & Thelwall (2012); Zahedi, Costas &
Wouters (2013); Mohammadi & Thelwall (2013);
Schlögl et. al. (2013); Haustein et.al. (2013a & 2013b)
Altmetrics & Citation as predictors:
Wardle, 2010
Eysenbach, 2011
Waltman & Costas, 2013
3

5. Objectives & Research Questions:
To distinguish patterns in terms of impact depending
on the types of Mendeley users
Q1. What do the different Mendeley users read in
terms of document types and Subject fields?
Q2. To what extent do the readerships of the
different users in Mendeley correlate with citation
indicators?
Q3. What is the impact of publications read by
different users in Mendeley?
4

6. Methodology (1)
Random Samples:
1. 20,000 WOS publications from all disciplines
between 2005-2011
2. 200,000 WOS publications from all disciplines
between 2011-2012
Metrics: Mendeley & Impact Story APIs
5

7. Methodology (2)
Collecting altmetrics on the basis of DOIs of the publications
Using Mendeley & Impact Story APIs
Linking and matching with WOS
Adding bibliometric indicators
Analyzing the data
6

8. Types of Mendeley users
• Professors (Associate, Assistant)
• Lecturers (Senior)
• Postdocs
• Researchers (Academics/non-Academics)
• Students (Bachelor, Master, Postgraduate)
• PhD/Doctoral
• Librarian
• Other professionals
• Unknown
7

9. Distribution of readerships in the
samples by types of users
8
Sample 1 Sample 2
34%
25%
17%
10%
7%
4%
3% 0,9% 0,4%
33%
28%
13%
10%
7%
5%
3% 1% 0,3% PhD
Unknown
Students
PostDocs
Researchers
Professors
Other
Professionals
Lecturer
librarians

10. Modeling impact by Mendeley users:
Scientific: Professors, PhD, Postdocs, Academic
Researchers
Educational: Lecturers, Bachelor, Master &
Postgraduate Students
Professional: Librarians, Other Professionals,
non Academic Researchers
Unknown: unidentified users
9
Sample 2
Sample 1
53%
14%
5%
28%
SCIENTIFIC
READERS
EDUCATIONAL
READERS
PROFESSIONAL
READERS
Unknown
52%
18%
5%
25%

11. What document type are more read by the
different users? (sample 1)
10
0%
5%
10%
15%
20%
25%
30%
35%
40%
Articles Reviews Non Citables Letter
PhD
Unknown Professions
Students
PostDoc
Researchers
Professors
Other Professionals
Lecturers
Librarians

12. Which fields are more read by types
of users?(sample 1)
11
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
PHD
Unknown
Students
PostDocs
Researchers
Professors
Other
Professionals
Lecturers
librarians

13. Which fields are more cited/read per
publication?(sample 1)
12
0
5
10
15
20
25
30
35
40
45
Readers per Paper (RPP)
Citations per Paper (CPP)

14. Which fields are more cited/read per
publication? (sample 2)
13
0
1
2
3
4
5
6
7
8
Readers per Paper (RPP)
Citations per Paper (CPP)

15. To what extent do the different types
of users in Mendeley correlate with
citation indicators?
Correlation
Readers Unknown PhDs PostDocs Students Researchers
Professors Other
Professional Librarian Lecturers
Sample
1
Citations
0,52 0,51 0,46 0,43 0,34 0,15 0,09 0,02 -0,01 -0,01
Sample
2 0,35 0,33 0,29 0,24 0,22 0,1 0,03 0,04 -0,01 -0,01
14
Correlation
Readers Unknown Scientific Educational Professional
Sample 1
Citations
0.52 0,51 0,48 0.34 0,05
Sample 2
0,35 0,33 0,30 0.21 0.07

16. What are the impact of publications
read by different types of readers?
15
0%
5%
10%
15%
20%
25%
Sample 1
sample2
PP Top 10%

17. Limitations
• Access only to the top 3 categories of readers in
Mendeley
• Data collection (time consuming)
• Speed of use the APIs (API limit)
• Scalability (limitations for the medium-large scale
analysis)
• Not perfect data matching with WOS (DOIs, ….)
16

18. Conclusions & Discussions
• Potential advantage of Mendeley over citations:
– for publications from social sciences and humanities
– for recent publications [!]
• Scientific users are more correlated with citations
than educational and professional users
• The other users could help to identify other types
of impact: educational, professional [?]
• Some users tend to read more highly cited papers
than others: Postdoc, PhD Students vs Professors
• Identifying the unknown users can shed some light
in detecting these other types of impact
• Further analysis needs to be done to dig into the
content of reading by different types of users 17

19. Thanks for your attention!
z.zahedi.2@cwts.leidenuniv.nl
18