This document discusses author-level bibliometrics and the multi-dimensional research assessment matrix. It presents the matrix, which outlines different units of assessment, purposes of assessment, output dimensions, and appropriate bibliometric and other indicators for evaluation. The document also provides examples of how bibliometric indicators can be used to operationalize different policy issues and measures. It notes challenges such as evaluating individuals based on collaborative work and the need for multiple indicators and expert review in research assessment.

1. Author
Level
Bibliometrics
Henk
F.
Moed
A
talk
at
a
Special
Session
on
Author
Level
Bibliometrics
at
ISSI2013
Vienna,
17
July
2013

2.
The
MulG-­‐Dimensional
Research
Assessment
Matrix
Expert Group on the Assessment of
University-Based Research (AUBR, 2010)

3. Multi‐dimensional Research Assessment Matrix (Part)
Unit
of
assessment
Purpose
Output
dimensions
Bibliometric
indicators
Other
indicators
Individual
Allocate
resources
Research
producGvity
PublicaGons
Peer
review
Research
group
Improve
performance
Quality,
scholarly
impact
Journal
citaGon
impact
Patents,
licences,
spin
offs
Department
Increase
mulG-­‐discipl.
research
InnovaGon
and
social
benefit
Actual
citaGon
impact
InvitaGons
for
conferences
InsGtuGon
Increase
regional
engagement
Sustainabi-­‐
lity
&
Scale
Internat.
co-­‐
authorship
External
research
income
Research
field
PromoGon,
hiring
Research
infrastruct.
citaGon
‘presGge’
PhD
com-­‐
pleGon
rates

4. Multi‐dimensional Research Assessment Matrix (Part)
Unit
of
assessment
Purpose
Output
dimensions
Bibliometric
indicators
Other
indicators
Individual
Allocate
resources
Research
producGvity
PublicaGons
Peer
review
Research
group
Improve
performance
Quality,
scholarly
impact
Journal
citaGon
impact
Patents,
licences,
spin
offs
Department
Increase
mulG-­‐discipl.
research
InnovaGon
and
social
benefit
Actual
citaGon
impact
InvitaGons
for
conferences
InsGtuGon
Increase
regional
engagement
Sustainabi-­‐
lity
&
Scale
Internat.
co-­‐
authorship
External
research
income
Research
field
PromoGon,
hiring
Research
infrastruct.
citaGon
‘presGge’
PhD
com-­‐
pleGon
rates
Read
column-­‐
wise

5. Multi‐dimensional Research Assessment Matrix (Part)
Unit
of
assessment
Purpose
Output
dimensions
Bibliometric
indicators
Other
indicators
Individual
Allocate
resources
Research
producGvity
PublicaGons
Peer
review
Research
group
Improve
performance
Quality,
scholarly
impact
Journal
citaGon
impact
Patents,
licences,
spin
offs
Department
Increase
mulG-­‐discipl.
research
InnovaGon
and
social
benefit
Actual
citaGon
impact
InvitaGons
for
conferences
InsGtuGon
Increase
regional
engagement
Sustainabi-­‐
lity
&
Scale
Internat.
co-­‐
authorship
External
research
income
Research
field
PromoGon,
hiring
Research
infrastruct.
citaGon
‘presGge’
PhD
com-­‐
pleGon
rates

6. Base
noGon
The
choice
of
indicators
depends
upon:
• What
unit
is
to
be
assessed
• Why
is
the
assessment
done?
• Which
aspect
is
being
assessed?
Indicators
that
are
appropriate
in
one
type
of
assessment
may
be
inappropriate
in
another

7. Valuable
noGons
• Data
accuracy
is
crucial
• Use
data
verified
by
authors
themselves
• Combine
metrics
and
expert
knowledge
• Impact
factors
no
subsGtutes
of
actual
impact
• Use
mulGple
indicators
• Take
into
account
phase
of
career
• Take
into
account
unintended
effects
• Present
“interpretaGon
manual”,
pros
&
cons
• Focus
on
top
vs.
bo_om
of
quality
distribuGon
• RetrospecGve
vs.
prospecGve
use
of
metrics

8. A
base
problem
• How
to
assess
the
performance
of
an
individual
by
analyzing
products
that
are
the
result
of
teamwork?
• Most
research
arGcles
are
the
result
of
collaboraGon
within
or
between
teams

9. More
and
more
individual
researchers
have
to
be
objecGvely
evaluated
for
promoGon
and
hiring
Use
easily
available
metrics
Count
total
publicaGons
or
#
arGcles
in
journals
with
the
highest
impact
factors
in
the
field
Meta
level
:
Policy
issue
Policy
measure
Bibliometric
operaGonali-­‐
zaGon
c
c
c
c
CASE
1

10. Research
community
is
not
sufficiently
oriented
toward
internaGonal
networks
SGmulate
publicaGon
in
good
internaGonal
journals
Count
#
arGcles
in
first
impact
quarGle
of
journals
in
subject
field
Meta
level
:
Policy
issue
Policy
measure
Bibliometric
operaGonali-­‐
zaGon
c
c
c
c
CASE
2

11. Professors
are
legally
bound
to
do
research
but
many
of
them
don’t
Allow
only
research
acGve
professors
to
decide
on
recruitment
of
new
research
staff
Select
only
professors
with
>=
3
publicaGons
in
7
years
in
recruitment
commi_ees
Meta
level
:
Policy
issue
Policy
measure
Bibliometric
operaGonali-­‐
zaGon
c
c
c
c
CASE
3

12. Wider
issues
• What
is
an
acceptable
“error
rate”
in
the
assesment
process?
• Wrong
in
individual
cases

benificiary
for
the
system
as
a
whole
• What
is
a
fair
assessment
process?