1.
CORE Analytics Dashboard
Petr Knoth
Jozef Harag
Drahomira Herrmannova
June 13, 2019 – OR 2019, Hamburg, Germany
CORE
Big Scientific Data and Text Analytics Group
Knowledge Media Institute, The Open University

2.
Introduction
• Research metrics becoming a key component of evaluation
processes
• Application of research indicators controversial, but demand for
them is increasing
• Universities want to understand and monitor the impact of their
research outputs
• This is evidenced by subscriptions to expensive products like
Elsevier’s SciVal, Clarivate’s InCites
• Typical cost: negotiated, it is believed £35-£50k per annum per
institution

3.
Existing solutions
• Elsevier SciVal
• Clarivate InCites
• Digital Science Dimensions

4.
Limitations of existing solutions 1/2
• Proprietary data (Scopus and Web of Science)
• Data or results of analyses cannot be easily downloaded and shared
with others
• Algorithms used for citation data acquisition not open/known
(Scopus and Web of Science)
• Complicates confirming validity/replicating results

5.
Limitations of existing solutions 2/2
• Data sparsity
• Scopus/Web of Science are known to have lower publication/citation
coverage than Google Scholar and Microsoft Academic [Harzing, 2017]
• Dimensions use primarily citations from I4OC which is an even sparser
citation dataset
• Lack of APIs for integration of results into existing systems
(Scopus and Web of Science)
• Slice-and-dice operations on titles/abstracts only
• Papers without DOI have metrics rarely available

6.
CORE Analytics Dashboard
• CORE is the largest full text Open Access aggregation service
[Notay, 2018]
• Because it is based on Open Access data, it can offer/enable:
• Slice-and-dice operations based on full text
• Full access to underlying data
• API integration with internal university systems

7.
CORE Analytics Dashboard
• A tool for analytical evaluation of universities’ research outputs
based primarily on CORE data
• Research impact insights on institutional level
• Comparison with other institutions or groups of institutions
• Based on citation data, social impact data
• Enables users to design and create custom graphs and add
these to their user area
• Intended for university research officers, repositories, university
management

8.
Use cases
• A university's Research Office manager wants to see how
her institution compares to another institution
• A university's Research Office manager wants to have an
overlook on research impact of a hot-topic (e.g. Artificial
Intelligence, DNA mapping, global warming, etc.) across UK
universities
• A non-Russell Group university wants to demonstrate (for
marketing reasons) that they have the same research and
social impact (or perhaps better) as any of Russell
Group universities in a given field.

9.
Data sources
1. CORE dataset
• Primary pivot, provides publication metadata, affiliation information
2. Microsoft Academic Graph
• Citation data
3. Crossref event data
• Publications’ Wikipedia and Twitter mentions
4. Mendeley
• Readership data, i.e. information about how many Mendeley users
have added a certain publication in their Mendeley library

10.
Data
• Data from across different systems were matched using DOIs
• Prototype version:
• UK institutions
• Outputs with a DOI
• Long-term goal is to expand to the whole word

11.
Interface overview
• Individual for each user
• Each user assigned a specific
institution
• Customizable user area:
users can define metrics and
graphs to track & display
• Institutional ranking

12.
User interface: statistics bar
• Each user assigned a specific institution
• Quick summary overview of institution’s performance

13.
User interface: dashboard
• Customizable area, user-defined charts

14.
User interface: institutional ranking
• Ranking according to a number of metrics

15.
Customizable area
• Enables creating user-defined
graphs
• Chart data can be
downloaded:
• Original (raw) data
• Aggregate chart data
• Chart PNG
• Full customizable, enables
resizing and reorganizing
charts

16.
User area: customizing the dashboard
• A step-by-step tool (benchmarking wizard) for creating custom
visualizations tailored to user’s needs
• Works in three steps:
1. Define publications sample filtered to select search criteria
2. Define what indicators to inspect
3. Select visualization chart to apply on the filtered data

17.
User interface: benchmarking wizard
1. Filter data – user specifies which articles to work with
• E.g. using keywords or phrases of interest that will be searched in
publications’ titles, abstracts, and full texts
• Also enables filtering, currently by year and publication type (research
article, thesis, etc.)
2. Selecting metrics to track – user defines which institutions to
compare and based on what metrics to compare them
3. Designing the chart – select chart type and name the chart
• Several chart types available including line charts, bar charts, scatter
plots, etc.
• User can customize colors, names of axes, etc.

18.
User interface: benchmarking wizard
1. Filter data – user specifies which articles to work with

19.
User interface: benchmarking wizard
1. Filter data – user specifies which articles to work with

20.
User interface: benchmarking wizard
2. Selecting metrics to track

21.
User interface: benchmarking wizard
3. Designing the chart

22.
Conclusions
• We presented CORE Analytics Dashboard – a tool designed to
enable users to analyse and compare the performance of
research outputs between universities along a variety of metrics
• Key difference from existing solutions – focus on collecting
performance indicators from openly available sources
• Goal – add a layer of transparency to research evaluation
• Goal – extend to whole world and add additional features
• E.g. collaboration graphs, UK REF predictions [Pride, 2018]

23.
Feedback
• www.menti.com
• Code 23 36 84

24.
References
• Harzing, Anne-Wil, and Satu Alakangas. (2017). "Microsoft
Academic is one year old: The Phoenix is ready to leave the
nest." Scientometrics 112.3: 1887-1894.
• Notay, Balviar. (2018). "CORE becomes the world’s largest
aggregator". Jisc scholarly communications.
https://scholarlycommunications.jiscinvolve.org/wp/2018/06/01/c
ore-becomes-the-worlds-largest-aggregator/. Accessed: 2019-
01-16.
• Pride, D. and Knoth, P. (2018). "Peer review and citation data in
predicting university rankings, a large-scale analysis". Lecture
Notes in Computer Science. Springer.

25.
Thank you!
Demo: bit.ly/core-analytics-dashboard