Key Performance Indicators of Research Programs

1. Exploring Emerging
Research Areas
Christine Burgess, PhD Anthony Molisani, PhD, MPH
October 17, 2017

2. 2
Key Performance
Indicators of Research
Programs
Through the Lens of
Emerging Research
Areas

3. 3
Today’s Topics
10 min
15 min
20 min
15 min
The importance of monitoring and evaluating
emerging research areas
Clarivate Analytics Research Fronts
How Research Fronts can be used to address common
program management questions
Q&A Session

4. The importance of monitoring
and evaluating emerging
research areas

5. 5
“We hear more and more these days about a
so-called crisis in scientific literature…There is
a literal deluge of literature published each
year, and it is increasing in geometric
proportions.”
- Dr Eugene Garfield, 1955

6. 6
With 2.5 million scientific articles published each year,
it’s becoming more complex staying on top of where science is going and who’s leading the way.
(Plume & van Weijen, 2014)
Increasing
numbers of
predatory journals2
Increasing
numbers of
predatory journals2
On average,
scientists read 264
articles per year3
On average,
scientists read 264
articles per year3
Number of
scientists globally
increases 4-5%
per year1
Number of
scientists globally
increases 4-5%
per year1
1
http://www.stm-assoc.org/2015_02_20_STM_Report_2015.pdf
2
http://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-015-0469-2
3
http://www.nature.com/news/scientists-may-be-reaching-a-peak-in-reading-habits-1.14658

7. 7
Funding for programs is limited
With research funding
capacity trending downward,
it is critical to be able to
understand how your
allocated funding is driving
research in your field
forward.
http://faseb.org/Science-Policy--Advocacy-and-Communications/Federal-Funding-Data/NIH-Research-Funding-Trends.aspx

8. 8
IDENTIFY EMERGING RESEARCH EFFECTIVELY
So, how do you find…
•topics that represent the leading edge of your field?
•which topics integrate knowledge from different research fields?
•contributors to cutting-edge science relevant to your research projects?
•the global hotbeds for a particular topic of interest (e.g., region, country)?
•how research is changing in your area?
?

9. Clarivate Analytics Research
Fronts

10. 10
At its core: Highly Cited Papers

11. 11
At its core: Highly Cited Papers
Top 1% in its field and year

12. 12
Identifying Research Fronts
Highly cited
papers from the
past five years
Papers that cite at
least one highly
cited paper

13. 13
Identifying Research Fronts
When researchers repeatedly
connect two papers in the
reference lists of the papers they
publish, there is typically a close
cognitive relationship between
the co-cited papers.
A Research Front consists of a
group of highly cited papers that
have been frequently co-cited, or
cited in pairs.
Derek J. de Solla Price Science, 149 (3683): 510-515, July 30, 1965.
Henry Small, Journal of the American Society for Information Science, 24 (4): 265-269, July-August 1973.
Henry Small and Belver C. Griffith, Science Studies, 4 (1): 17-40, 1974.
Henry Small, Journal of the American Society of Information Science and Technology, 54 (5): 394-399, March 2003.
Henry Small, Scientometrics, 68 (3): 595-610, December 2006.

14. 14
Identifying Research Fronts
Research
Front
Research
Front
Core Papers Citing Papers

15. 15
Identifying Emerging Research Areas
Emerging Research Areas can vary based on
the definition and priorities of an entity and
may be identified through a variety of
methodology
A method in which Clarivate Analytics
employs Research Fronts to help identify
emerging research areas is to include the core
Research Front together with its citing papers
that define the cohesive relationship among
the core.
The citing papers represent the most recent
work and therefore represent the leading or
emergent edge of the Research Front.

16. 16
There are ~8,000-10,000 Research Fronts which take time to sort through
Processes and methodology are in place to
facilitate and extract the right information

17. How Research Fronts can be
used to address common
program management
questions

18. 18
Using Research Fronts
The critical value of Research Fronts methodology comes from the ability to attribute additional metrics to
emerging research areas using the papers that comprise it to provide context and utility.

19. 19
Funding
Clarivate Analytics captures all funding
acknowledgements from publication beginning in
2008. Funding and grant data from repositories such
as RePORTER can also be used and linked to
publications within the emerging research area.
.
Funding of the core papers indicates when
support is associated with the highly cited
papers at the heart of each emerging
research area.
Funding of the citing papers
indicates how your program is
supporting the overall
emerging research areas or
how sponsored research is
building on the core papers.
To what extent has my program
contributed funding to this emerging
research area?
Which funders are most active in funding
the core, leading edge, or both areas of an
emerging research area?

20. 20
Identifying Emerging New Fields
Emerging new fields can form within the boundaries of
an existing discipline or outside existing fields. Identifying
the potential for an emerging new field to represent the
beginning of a new field helps ensure that new research
areas are receiving sufficient support.
The potential for a Research Front to represent the
emergence of a new field can be determined by
understanding how the core research papers are
integrating knowledge from different research fields
using the Porter’s Integration Index (I).
Porter, A.et al. Measuring Researcher Interdisciplinarity. 2007. Scientometrics. (72,1, 117-147)
Buter, R. et al, Identification of converging research areas using publication and citation data 2010
Research Evaluation (19,1, 19-27)
Which emerging research areas integrate
source material from the most diverse
group of subject categories?
Which institutions have supported this type
of interdisciplinary research?

21. 21
Understanding Translational Movement
The translational nature of an emerging research
area can be determined by mapping the papers
contained within it to the Weber Biomedical
Triangle by using the paper’s MeSH terms.
Citing Papers
Core Papers
Translational
Movement
Weber, G. Identifying translational science within the triangle of 2013. J Transl Med. (11: 126.)
Which emerging research areas in my field
are on the threshold of moving into clinical
space?
How is the research in this emerging area
moving towards or away from the clinic
over time?

22. 22
Understanding the Emerging Research Areas Landscape
Jaccard P. (1908) Nouvelles recherches sur la distribution florale. Bull. Soc. Vaud. Des. Sci. Nat ., 44, 223–270.
For example, the citing papers of the emerging research areas are evaluated using the Jaccard Index to
determine the ratio of intersection over the union of papers within emerging research areas.
What are the relationships of the research
between emerging areas?
Which emerging research areas are the
weak ties bridging multiple clusters of
research topics?

23. 23
Regional Activity
The listed affiliations of authors on a
publication are used to identify where
the core and citing research of an
emerging research area is being
conducted and to what extent.
What are the geographical hotbeds within
an emerging research area?
Who are the organizations and institutions
best positioned for collaboration and/or
funding?

24. Understanding the impact of
NIH and NSF funded research
on a specific emerging
research area based on
keywords.
Use case: Investing in the core of emerging
research

25. 25
Use case: Investing in the core of emerging research

26. 26
Use case: Investing in the core of emerging research
8 of 12 core papers
acknowledge funding
from NIH + NSF
Research Front:
Hydrogels with tunable stress relaxation regulate stem cell fate and activity

27. 27
Use case: Investing in the core of emerging research
46% of citing
papers
acknowledge
funding from
NIH+NSF
Funding for 66% of core papers in the Research Front contributed to
the 1,030 emerging research papers of:

28. 28
Use case: Investing in the core of emerging research
In a brief analysis of one
Research Front of interest,
we can already begin to
identify the role and impact
NIH+NSF funding is playing
on Emerging Research Areas

29. Christine Burgess, PhD Managing Consultant +1(240) 618.1302 christine.burgess@clarivate.com clarivate.com
Anthony Molisani, PhD, MPH Presales Scientist +1(215) 823.1773 anthony.molisani@clarivate.com clarivate.com