The Royal Irish Academy and the Irish Research Council held a forum on Open Access in May 2013. More Details - http://www.ria.ie/about/our-work/policy/ria-initiatives/making-open-access-work-for-ireland.aspx

1. Dr Julia M WallaceDr Julia M Wallace,
Project Manager – PEER
julia@iglooe comjulia@iglooe.com
RIA & Irish Research Council Forum:
Making Open Access Work for IrelandMaking Open Access Work for Ireland
2 May 2013
Findings from the PEER ProjectFindings from the PEER Project
Supported by the EC eContentplus programme
PEER − Publishing and the Ecology of European Research 1 www.peerproject.eu
Supported by the EC eContentplus programme

2. Project origins & objectivesProject origins & objectives
• Arose from impasse about embargo periods on Green open access• Arose from impasse about embargo periods on Green open access
• PEER was set up to monitor the effects of systematic archiving
of ‘stage two’ research outputs (NISO: accepted manuscripts)g p ( p p )
– Large-scale ‘experiment’ regarding deposit of author manuscripts in an
‘observatory’ of OA repositories
R h t di i i d t th h d id t i f– Research studies commissioned to gather hard evidence to inform
future policies: Usage, Behavioural, Economic
– Collaborative project of diverse stakeholder groups: Publishers,p j g p
research community and library/repository community
• Duration: 09/2008−05/2012 (3 years plus 9 months extension)
B d t/F di €4 2M 50% f th EC ( C t t l ) 50% t• Budget/Funding: €4.2M : 50% from the EC (eContentplus programme) 50% partners
• Executive partners: STM (coordination), ESF, UGOE/SUB, MPG/MPDL, Inria
(Technical partners: Uni Bielefeld, SURF)
PEER − Publishing and the Ecology of European Research 2 www.peerproject.eu

3. Participating Publishers, Repositories &
LTP hiLTP archive
PEER − Publishing and the Ecology of European Research 3 www.peerproject.eu

4. The PEER Observatory & content levelsThe PEER Observatory & content levels
Publishers: 241
Publishers
submit 100%
Publishers invite
authors
11,800
invitations
Publishers invited
Eligible participating
journals
metadata
Authors Self-
deposit
>53 000 mss
invitations
170 mss
Europe based
"PEER authors" to
participate in
survey for
Publishers submit
50% + manuscripts
Central Deposit
interface
>53,000 mss behavioural research
Publishers/ repositories
delivered usage data
100% EU
> 22,500 EU mss
delivered usage data
(log files) for
usage research
/manuscripts
& metadata
LTP:KB
eDepot
PEER REPOSITORIES
Embargo expired
Publishers/ repositories
queried for
economics
research
SSOAR MPG
HAL ULD TCD
UGOE
Embargo expired
~19,000 by project end
>20,000 mss today
PEER − Publishing and the Ecology of European Research 4 www.peerproject.eu

5. PEER Challenges and Solutions (1)
PUBLISHER CHALLENGES PUBLISHER / PEER DEPOTPUBLISHER CHALLENGES
• Stage two (accepted manuscripts) not
standard extraction point
PUBLISHER / PEER DEPOT
SOLUTIONS
Change Process at Publishers
• Author accepted manuscripts in a
variety of file formats
Only one file format allowed – PDF
Checking mechanisms: journal/
• All article types submitted
Checking mechanisms: journal/
article
ISSN check
article type check
Article kept until metadata
• Metadata delivery in several batches
– Article metadata are incomplete at acceptance
time; Publication date unknown, DOI not
attributed
Extraction of only EU“ authored manuscripts not
Article kept until metadata
completion
Metadata are accepted in either
one step (on publication) or two passes (on
acceptance and on publication)– Extraction of only „EU authored manuscripts not
possible at acceptance stage
• Different metadata formats
NLM2 x NLM 3 0 ScholarOne proprietary
acceptance and on publication)
EU author filter done at PEER Depot
Mapped onto single TEI structure
– NLM2.x, NLM 3.0, ScholarOne, proprietary
• Some Metadata elements delivered
within PDF document
Extraction done at PEER Depot
(GroBID) in order to increase
content
PEER − Publishing and the Ecology of European Research 5 www.peerproject.eu

6. PEER Challenges and Solutions (2)
REPOSITORY CHALLENGES REPOSITORY / PEER DEPOTREPOSITORY CHALLENGES
• Varying metadata requirements
REPOSITORY / PEER DEPOT
SOLUTIONS
Convert TEI metadata into internally
d t d t t d d
• Varying ingestion processes
used metadata standard
Implement SWORD protocol for
transfer between Depot &
• Hosting PEER content
repositories
Build dedicated PEER Repository
within framework of home institution
• Not configured for accurate embargo
management
Embargo management undertaken
at PEER Depot (0 - 36 months)
• Author authentication
• Logfile provision
Central deposit interface at MPDL
then transfer to PEER Depot
Set up anonymisation process plusog e p o s o p y p p
automated transfer to Usage team
An Additional PEER Challenge: Stage two versions were ‘messy’ – requiring
l i t ti i h i / th t i l d d
PEER − Publishing and the Ecology of European Research 6 www.peerproject.eu
manual intervention in cases where reviewer / author comments were included

7. Behavioural research: Final ReportBehavioural research: Final Report
Department of Information Science and LISU at Loughborough
University, UK (Two phases of Research between 2009 and 2011)University, UK (Two phases of Research between 2009 and 2011)
• Researchers who associated Open Access with ‘self archiving’ were in the minority.
(Physical sciences & mathematics and the Social sciences, humanities & arts - green; Life
sciences and the Medical sciences - gold)g )
• Anecdotal evidence that some researchers consider making journal articles
accessible via Open Access to be beyond their remit. (Also PEER experience)
• Authors tend to be favourable to Open Access [ ] with the caveat that self archiving• Authors tend to be favourable to Open Access [..] with the caveat that self archiving
does not compromise the pivotal role of the published journal article
• Readers have concerns about the authority of article content and the extent to which
it can be cited when the version they have accessed is not the published finalit can be cited when the version they have accessed is not the published final
version.
• Academic researchers [..] do not desire fundamental changes in the way research is
currently disseminated and publishedcurrently disseminated and published.
• Open Access Repositories are perceived by researchers as complementary to, rather
than replacing, current forums for disseminating and publishing research
PEER − Publishing and the Ecology of European Research 7 www.peerproject.eu

8. PEER Usage ResearchPEER Usage Research
CIBER Research Ltd, UK
• High volume of content in the project (>18,000 EU deposits publicly available
March 2012) supporting research with a high degree of confidence
• Measure activity over 12 months starting March 2011• Measure activity over 12 months, starting March 2011
• Log file collection & analysis 1 March - 31 August 2011
• Randomised Controlled Trial: suppression of 50% PEER content at partner
f & 1 2011 29 2012repositories. Logfile collection & analysis 1 December 2011 – 29 Feb 2012
Caveats:Caveats:
The ‘PEER’ system is unlikely to have reached ‘steady state’ when study
undertaken
Observations of effects within PEER cannot be extrapolated to the OpenObservations of effects within PEER cannot be extrapolated to the Open
Access universe
PEER − Publishing and the Ecology of European Research 8 www.peerproject.eu

9. ‘No effect’ publisher hypothesis: key findingsp yp y g
Making preprints visible in PEER is associated with more traffic to
the publisher sites (significant if relatively modest increase)the publisher sites. (significant, if relatively modest increase)
Publisher full text downloads increased by 11.4%
95% confidence intervals: 7.5% to 15.5%, highly statistically significant at p <0.01g y y g p
Publisher downloads went up in all subject areas, but with variation:
Statistically significant increases in life sciences: up 20.3% (13.1% to 27.9%,
p<0.01) ; physical sciences: up 13.1% (5.2% to 21.6%, p<0.01)p ) p y p ( p )
Statistically insignificant findings in medicine: up 5.2% (-1.0% to 11.7%,
p=0.10) ; social sciences and humanities: up 4.1% (-0.05% to 13.9%, p=0.38)
PEER − Publishing and the Ecology of European Research 9 www.peerproject.eu

10. PEER Usage Research: results & observationsPEER Usage Research: results & observations
• The likely mechanism is that PEER offers high quality metadata (including
DOI) allows a wider range of search engine robots to index its content thanDOI), allows a wider range of search engine robots to index its content than
the typical publisher, and thus helps to raise the digital visibility of scholarly
content.
• There are variations as we zoom in on the detail and the jury is still out in
medicine, the social sciences and humanities, and for smaller publishers,
for reasons we do not understand yet.y
Results suggests that deposit in institutional repositories doesn’t seem to
cause harm WITH JOURNAL SPECIFIC EMBARGOS
• But, PEER does not tell us about the effects of deposit in SUBJECT
repositories. Recent research suggests a 14% reduction in usage statistics
from journal websites for research articles systematically deposited in PMCfrom journal websites for research articles systematically deposited in PMC
with 12 month embargo - NIH mandate. (See P Davis article in The Physiologist:
http://www.the-aps.org/mm/Publications/Journals/Physiologist/2010-
present/2012/October.pdf)
PEER − Publishing and the Ecology of European Research 10 www.peerproject.eu
p p )

11. PEER Economics Research: Conclusions
ASK research centre, Bocconi University, Milan, Italy
First detailed empirical study of cost drivers of publishers and
repositories (22 organizations)
Cost structure of journals (excluding overheads):
• Management of peer review: 250 USD per submitted article – nog p p
economies of scale
• ‘Production’ costs : 170 - 400 USD (formatting, editing,
typesetting, metadata)
Cost of content management:
• Investment in digital platform from USD 1 million up
• Maintenance costs USD 170 k– 400k
PEER − Publishing and the Ecology of European Research 11 www.peerproject.eu

12. Repositories: A lean cost structure ….Repositories: A lean cost structure ….
• Cost of processing documents (including metadata creation)
10 EUR max per reference10 EUR max per reference
18 EUR max per full text
43 EUR max per journal article43 EUR max per journal article
• Positive impact of harvesting and mandates in the speed up of
feeding processg p
• Set up cost of repository was not determined;
• Cost of technical FTE per itemp
between 2 and 50 EUR per reference
between 2,5 and 53,2 EUR per full text journal article, , p j
…. with a big impact of sunk and
organizational costs
PEER − Publishing and the Ecology of European Research 12 www.peerproject.eu
organizational costs

13. Executive Partners- Points of AgreementExecutive Partners- Points of Agreement
• Building a large-scale infrastructure is organizationally and• Building a large-scale infrastructure is organizationally and
technically challenging — even at a project level
• Author self-archiving is unlikely to generate a critical mass of Greeng y g
OA content
• Version II archiving requires considerable manual oversight and
i t tiintervention
• Scholars prefer the Version of Record
• Usage scenarios for Green Open Access are more complex than
generally acknowledged
Th t d tilit f bli hi (“G ld”) h• The acceptance and utility of open access publishing (“Gold”) has
increased rapidly (during the lifetime of the project)
PEER − Publishing and the Ecology of European Research 13 www.peerproject.eu

14. For further information on PEER including final
research reports and presentations made at the
PEER End of Project Conference visit:
www.peerproject.eup p j
PEER − Publishing and the Ecology of European Research 14 www.peerproject.eu