UBC OAweek 2012 presentation by Heather Piwowar. Covers research impact tracking, altmetrics, data sharing, ImpactStory, and more.

1. Uncovering the
Impact Story
of Open Research
Heather
Piwowar
@researchremix
Postdoc
with
NESCent
and
Dryad,
at
Duke
and
UBC
some photos NC, SA
UBC
2012

2. Research evaluation
matters

3. who gets hired
who gets grants
which journals succeed
which journal innovations succeed
whether we publish in journals at all
whether we actually write articles
the whole structure of scientific communication

4. right now, impact factor

5. problems!
run-away hits
decision of 4 anonymous people
only one type of impact
favours the established

6. let me tell you something that is
going to start changing the game

7. A few weeks ago, on Oct 4,
NSF announced it is going to change
its Biosketch Template to call for
“Products”
instead of
“Publications”

8. Big deal. So it is a new title.

9. Whole lot more than that.

10. 1.
A major science funder believes status quo isn't the only
way to disseminate research, and traditional peer-reviewed
articles in high impact journals are not all that matters.
Other research products can be considered
first-class research products.

11. 2.
It signals this to all the researchers who apply to the NSF.
This is a big deal, because it moves the discussion
away from just OA advocates and open data tweeps
to all scientists. All scientists now get to think,
"hrmm, did I release a dataset that is as good as my papers?"

12. 3.
It necessitates a move away from journal impact factors.
The quality and impact of data and software can't be
evaluated with the journal impact factor.
Item level metrics and altmetrics are going to be totally
necessary for grant reviewers to know whether these
alternative products have made a difference.

13. 4.
Once we are in the grant review door with item-level metrics,
watch out!
Alternative metrics are empowering for innovations in
publishing. All the new journal experiments that are
springing up... eLife, Peerj, F1000 Research, PLoS
Currents.... are invited to compete, invited to help authors
make the case that the articles published in those containers
were impactful.
Very empowering for innovative journals, most of which are
OA. And empowering for dismantling the "article" yet further
into its component parts.

14. 5.
It demonstrates a willingness on the part of funders to
refine what they consider evaluation-worthy criteria.
Transformation is going to happen as major funders
encourage their applicants to brag about their openness,
and the impact that their openness has enabled.

15. what can the world
look like

16. Beyond the
Impact Factor

17. Beyond the
Article

18. Quality
Impact

19. A Single Number
Multiple Dimensions

20. Multiple Flavours
CC-BY-NC by maniacyak on flickr
http://www.flickr.com/photos/maniacyak/3432589472

21. Open

22. Context

23. Agile
experimentation

26. Tell the full story of
research impact.

27. altmetrics.org/tools
Altmetric.com
ImpactStory
PLOS article-level metrics
Plum Analytics
Reader Meter
Science Card

28. http://www.flickr.com/photos/pixscapes/4331070047

29. born as total-impact
Hackathon!
incorporating as nonprofit
Board:
Cameron Neylon, John Wilbanks

33. http://orcid.org

42. what can the world
look like

43. Beyond the
Impact Factor

44. Beyond the
Article

45. datasets!
software!
slides!
preprints!
blogs!

46. context

47. type of engagement
type of audience

48. comparisons, reference sets

50. not just one number

51. not just one flavour

52. open!

54. rOpenSci
Plum
Ubiquity Press
PeerEv
dissertations
IR experiments
...

56. barrier-based $$$

57. http://www.flickr.com/photos/quinnanya/2055471833

58. http://flickr.com/photos/nieve44/2152712789/

60. Changes are happening in
scholarly communication.
http://www.flickr.com/photos/myklroventine/892446624/

61. http://www.flickr.com/photos/jo-h/2688026447/

62. Build great things.
Reward impactful work.
Tell the full story of
our research impact.

63. thank you!
Jason Priem: cofounder of ImpactStory
Also: Todd Vision, Mike Whitlock, the open science community, and
those who release their articles, datasets and photos openly.
blog.ImpactStory.org
team@ImpactStory.org
@ImpactStory
ImpactStory.org