1. Researcher Awareness + Perception:
A year in review
NISO Altmetrics Project Meeting
Washington, D.C. – December 11, 2013
Michael Habib, MSLS
Sr. Product Manager, Scopus
habib@elsevier.com
Twitter: @habib
http://orcid.org/0000-0002-8860-7565

2. About 1 year ago… October 2012
Background & approach:
54,442 individuals were randomly selected from
Scopus
3,090 respondents completed
Representative response by country and discipline.
Error margin
1.5%, at 90% confidence levels
Adrian Mulligan, Gemma Deakin and Rebekah Dutton
Elsevier Research & Academic Relations
2

3. 10/12: Most widely known by researchers
Impact Factor
82%
H-Index
43%
Journal Usage Factor
10%
SJR
4%
Altmetrics
???
Impact Factor is published by Thomson Reuters, Altmetrics were least well known
3

4. 10/12: Most widely known by researchers
Impact Factor
82%
H-Index
43%
Journal Usage Factor
10%
SJR
4%
Altmetrics
1%
4

5. One year on? Most widely known by
researchers in Q3 (n=326)
Impact Factor
88%
H-Index
70%
Journal Usage Factor
14%
SJR
14%
Altmetrics
???
% Awareness of quality metrics (n=326, Q3 13) – From internal study by Elsevier Research &
Academic Relations - Mingxin Zhou / Cat herine Fielding-Huda - October 2013
5

6. One year on? Most widely known by
researchers in Q3 (n=326)
Impact Factor
88% (+6)
H-Index
70% (+27)
Journal Usage Factor
14% (+4)
SJR
14% (+10)
Altmetrics
5% (+4)
6

7. Generally metrics with the highest awareness are
also considered to be the most useful
Percentage of aware respondents that chose the
metric as one of the most useful
70%
Impact factor
60%
h-index
50%
40%
30%
The trendline shows the linear
trend for the relationship
between awareness and usage
of metrics
R² = 0.697
Altmetrics
Journal Usage Factor
F1000
SJR
Eigenfactor
SNIP
Metrics above the line have
lower levels of awareness, but
are more likely to be rated as
useful than the typical
awareness-usage relationship
20%
10%
0%
0%
10%
20%
30%
40%
50%
60%
70%
-10%
-20%
Metrics below the line have
higher levels of
awareness, but are less likely
to be rated as useful than the
80%
90%
typical awareness-usage
relationship
Percentage of respondents that are aware of the metric
* This is the 2012 data again See appendix for background and approach. Research by Elsevier
Research & Academic Relations. Impact Factor is published by Thomson Reuters,
7

8. 32 (10%) Scopus users stated they
know of Altmetric for Scopus. 25 of
them think it very useful or
somewhat useful.
% Awareness of Altmetric for Scopus
n = 326 (Q3 13)
Yes
10%
No
90%

9. Assessing the usefulness of potential quality 

metrics: by age
Significant difference between
subset and total (subset higher)
Significant difference between
subset and total (subset lower)
Q3 Thinking about possible new measures of research productivity, how useful do you think the below would be in assessing the quality of a
researcher or a research article?(By age)
% Think it would be extremely/very useful
Under 36 (n=540)
Article
views/downloads (for
articles)
43%
Citations from
materials that are in
repositories
Share in social
network mentions (for
articles)
Number of readers
(for articles)
Number of followers
(for researchers)
36-45 (n=920)
49%
44%


21%

Votes or ratings (for
articles)
35%

A metric that measures
the contribution an
individual makes to peer
review (for researchers)
34%

A score based on
reviewer assessment (for
articles)
33%

33%
24%
36%

43%
41%
15%
41%
37%

43%

12%
39%
28%
22%
TOTAL
(n=3,090)
Over 65 (n=242)
44%
41%

18%
56-65 (n=507)
45%
45%
42%
38%
46-55 (n=819)
13%
16%
41%

30%
22%
29%
27%
26%
29%
27%
27%
35%

31%
30%
19%
24%
27%
40%

24%
28%
28%
9

10. Assessing the usefulness of potential
quality metrics: by region (1 of 2)


Significant difference between
subset and total (subset higher)
Significant difference between
subset and total (subset lower)
Q3 Thinking about possible new measures of research productivity, how useful do you think the below would be in assessing the quality
of a researcher or a research article? (By region, slide 1 of 2)
% Think it would be extremely/very
useful
Africa
(n=72)
Article views/downloads (for
articles)
56%
Citations from materials that are
in repositories
Share in social network mentions
(for articles)
Votes or ratings (for articles)
A metric that measures the
contribution an individual makes
to peer review (for researchers)
A score based on reviewer
assessment (for articles)

Eastern Europe
(n=183)
50%
55%
51%

26%
Number of readers (for articles)
Number of followers (for
researchers)
APAC
(n=803)
40%
44%

50%

43%

49%

49%

43%

16%

19%

21%

46%
36%
33%
50%
46%
49%
29%
TOTAL
(n=3,090)


27%
Latin America
(n=182)
45%
41%

30%
28%

35%


36%

26%
40%
45%


31%
34%
24%
24%
28%
32%
35%

28%
10

11. Assessing the usefulness of potential quality

metrics: by region (2 of 2)
Significant difference between
subset and total (subset higher)
Significant difference between
subset and total (subset lower)
Q3 Thinking about possible new measures of research productivity, how useful do you think the below would be in assessing the quality of
a researcher or a research article? (By region, slide 2 of 2)
% Think it would be extremely/very
useful
Middle East (n=47)
North America (n=770)
Article views/downloads (for
articles)
40%
40%
42%
Share in social network
mentions (for articles)
19%
43%
Number of readers (for articles)
Number of followers (for
researchers)
Votes or ratings (for articles)
A metric that measures the
contribution an individual
makes to peer review (for
researchers)
A score based on reviewer
assessment (for articles)
32%
28%
36%
32%

10%
36%
23%
19%
TOTAL
(n=3,090)
11%

36%

43%

43%

41%
Citations from materials that
are in repositories
Western Europe (n=1,033)
16%

40%

23%

31%

22%

24%
32%
26%
23%

28%
34%
26%
22%

28%
11

12. Thank you!
Michael Habib, MSLS
Sr. Product Manager, Scopus
habib@elsevier.com
Twitter: @habib
http://orcid.org/0000-0002-8860-7565

13. Background & approach
Who & when: 54,442 individuals were randomly selected from Scopus.
They were approached to complete the study in October 2012. To ensure
an unbiased response Elsevier’s name was only revealed at the end of
the survey.
Responses: The online survey took around 15-20 minutes to complete.
3,090 respondents completed it, representing a response rate of 5.7%.
Data has not been weighted. There was a representative response by
country and discipline.
Statistical testing: Error margin 1.5%, at 90% confidence levels. When
comparing the score for main group and sub-groups we have used a Z test
of proportion to identify differences between the overall average and the
sub-group (90% confidence levels), when there are 30 or more responses.
Adrian Mulligan, Gemma Deakin and Rebekah Dutton
Elsevier Research & Academic Relations
13