Process and different systems of scientific peer review are presented.

1. Understanding scientific peer review
Samir Haffar M.D.
Assistant Professor of Gastroenterology

2. 1st documented description of
a peer review process
Ishāq bin Ali Al-Rahawi (854 – 931)

3. 1st documented description of
a peer review process
• This book states that it is the duty of a physician to make
duplicate notes of the patient’s condition on each visit.
• When the patient had been cured or had died, the notes of
the physician were to be examined by a local council of
physicians, who would assess whether the physician had
practiced according to the standard of that era

4. Peer review to adjudicate scientific publications
“Royal Society of London” in 1752

5. Peer review to adjudicate scientific
publications
• The “Royal Society of London” established in 1752
a committee to assess publications submitted to their
journal “Philosophical Transactions”
• The purpose was to help the editor choose appropriately-
themed articles and not to strictly assess their quality

6. Peer review as we know it now
• Began in the United States in the early 1900s
• First journals to implement it
Science
JAMA
American Practitioner

7. Introduction of photocopiers (1959)
Peer review process was facilitated by
introduction of photocopiers, which
allowed production and circulation of
multiple copies of submitted articles to
external reviewers without fear of losing
the original documents

8. • Assisting editorial decisions as to whether accept or
reject a paper
• Helping to improve the quality of submitted papers
Peer review is perceived as contributing in 2 ways

9. Peer review process

10. Systems of scientific peer review
• Single-blind peer review
Reviewers aware of authors’ identity but not vice versa
Norm in clinical medicine and biomedical journals
• Double-blind peer review
Authors & reviewers not aware of each other’s identity
Widely used in imaging, nursing and humanities journals
• Open peer review
Authors & reviewers known to each other
BMJ moved to an open review more than a decade ago
70 journals from BMC use variable models of open peer review
BMC: BioMed Central

11. Single-blind peer review
• Advantages Reviewers free from authors’ influence
Forthright in critiques without fear of conflicts
• Disadvantages Advantageous to well-known authors
Detrimental to junior authors
Discrimination against authors because of
nationality, native language, gender or institution

12. Double-bind peer review
• Advantages: Offering all advantages of single-blind peer review
May protect authors from reviewers’ bias
• Disadvantages: Certain abuse persist in single or double-blind review
Plagiarism of non published data for personal benefit
Deliberate delay of publication to usurp the authors
Take longer time: time to uncover author’s identity
They were often able to identify authors correctly
• Effect on quality: No difference in quality or decision to publish
in small trials between blinded/unblinded reviewers

13. Reasons for blinding failure
• Reviewers successfully identify the authors as often as
25-50% of the time in both biomedical & social journals
• Common reasons for blinding failure:
Reviewer familiarity with authors’ work
Authors inadvertently revealed their identity in manuscript

14. How can authors avoid blinding failure?
 Use of the first person
Change “we have shown that..” to “Jones et al. have shown that..”
 Excessive citations from a single author or institution
 Citations that are in press
 Use of the author’s initials in the manuscript: to add after acceptance
 Acknowledgments: can be written after acceptance
 Failure to remove names of institutions from images & drawings
 Use of figures from previous publications
Krinsky G. AJR 1999;172:1474

15. Open peer review
• Advantages: Increases transparency
Reviewers more constructive in their critiques
Could help prevent plagiarism
• Disadvantages: Possible animosities between authors & reviewers
Higher decline rate from reviewers to participate
Increase in time taken to write reviews
• Effect on quality: Contradictory and inconclusive
Despite the lack of evidence in favor of open reviewing,
many journals began to implement or experiment with this system.
70 journals from BMC now use variable models of open peer review

16. Peer review as a source of bias
Difficult to detect & quantify
• No significant results Can be denied publication
Go to multiple journals leading to delay
Published in low impact factor journals
Published in languages other than English
• Reviewers ask authors Delete or combine outcomes
Modify analysis
Perform post hoc subgroup analyses
These actions can introduce classical biases

17. Possible role of peer review to introduce bias

18. Which system of peer review is better?
• Difficult to answer the question with certitude based on present data
• Each system has advantages & challenges
• Both double blind & open peer reviews more supported at present
• Double-blind peer review appears less biased at face value
Burdens on editorial staff reduced by a checklist system & software

19. Peer review process is far from perfect
“Peer review process is expensive, slow, prone to bias,
open to abuse, anti-innovatory, and unable to detect fraud”
Dr. Richard Smith
Former editor of The BMJ

20. Fraud in peer-reviewed journals
• John Darsee (US cardiologist)
Considered a shining star in his field
Published over 100 papers & abstracts at Harvard & Emory
Colleagues were suspicious of his works & reported him in 1981
Withdrawn: Emory (8 papers, 32 absts), Harvard (9 papers, 21 absts)
• Diederik Stapel (Dutch psychologist)
Award from “Society of experimental Social Psychology”
Dean of social and behavioral sciences faculty/Tilburg university 2010
The University suspended Staple's data in September 2011
At least 55 publications in peer-reviewed journals were affected

21. Despite the continued criticism of peer review,
few would vote for its abolition,
and most would advocate for its improvement,
so more research is needed
Rennie D, Flanagin A. Three Decades of Peer Review Congresses.
JAMA 2018;319:350-53.

22. How to improve the peer review process?

23. Groups involves in peer review
• Authors Wish to publish their work for career promotion
or secure funding
• Editors Under pressure to identify novel & sound research
• Reviewers Try to fit a time-consuming process with busy schedules
& often without remuneration or academic credit
• Publishers Compete in a transforming landscape of fast and
abundant science publishing
All these groups should be engaged collectively to
improve the peer review process
https://doi.org/10.1016/j.pan.2017.08.012

24. Role of authors
• Should adhere to instructions set by journals when they
prepare their manuscripts
• Penelope* is an automated tool that checks manuscripts and
gives immediate feedback to authors before submission
• Would be sound to allow authors the right to appeal against
decision of editors & reviewers when they find it to be unfair
* https://www.penelope.ai

25. Role of editors
• Few studies of editors’ behaviors have been made,
and little is known beyond what they reveal themselves
• A judicious editor can play an adjudicating role and
prevent many of the potential abuses of reviewers
such as hostile comments, unsubstantiated criticisms, and
delay of competitors’ manuscripts
• Some argue for code of practice for editors which could
help1
(1) Newton DP. Accountability in research 2010;17(3):130-45.

26. Role of reviewers
• Reviewers trained in epidemiology & statistics, of younger
age, and who spent up to 3 hours on a review, have been shown
to be associated with higher review quality in one study
• Using standard checklists by reviewers rather than giving a
report in their own words could be helpful
• Publons* a platform focusing on peer review, has developed
Publons Academy which organizes practical peer review
training courses for early career researchers
* https://publons.com/home/

27. Review Quality Instrument (Version 3.2)
van Rooyen S et al. J Clin Epidemiol 1999;52:625–62.

28. Review Quality Instrument (Version 3.2)
van Rooyen S et al. J Clin Epidemiol 1999;52:625–62.

29. Checklist to review manuscripts
Only for guidance – reviewers not obliged to answer all questions
Title Is the title informative? Is it too long? Does it relate to content of article?
Key words Are the keywords appropriate? Do they reflect the content of the article?
Abstract Is the abstract structured? Is there an aim and a hypothesis?
Introduction Do authors explain background of problem & list recent relevant studies?
Materials/methods Is the materials and methods section structured?
Are following subheading used: Study design, Subjects, Methods,….
Subjects Do authors follow the recommended reporting guidelines for their study?
Are precise inclusion and exclusion criteria provided?
Methods Are methods explained in sufficient detail?
Statistical analysis Do the authors explain their rationale for using different tests?
Are the name, version and manufacturer of statistical programs provided?
Results Do the authors explain any missing values?
Are the tables informative? Are column & row titles informative?
Discussion Do the authors describe the limitations of their study?
Do the authors draw clear conclusions based solely on their results?
References Are the references up-to-date and formatted according to journal style?
Smolčić VŠ, Šimundić AM. Biochemia Medica 2014;24(3):321–8

30. Role of publishers
• There is too little sound research on peer review
• The American Association for the Advancement of Science,
publisher of Science, announced the PRE action which makes
peer review material available alongside published papers
• Lee and Moher, specialists in publication science at University of
Washington and Ottawa Hospital Research Institute, call publishers
to enable opening of what they name “black box of peer review”
It is now time to conduct systematic experimental studies to assess
various peer review practices and improve quality & transparency
PRE: Peer Review Evaluation

32. You can read reviewers’ comments & authors’ reply

33. International congress on peer
review and scientific publication
• Launched by American Medical Association and British
Medical Association in 1986
• The congress is held every 4 years
• Activities of this congress are reported in special themed
issues of JAMA
http://www.peerreviewcongress.org

34. 8th International congress on peer review and
scientific publication (Chicago - Sep 2017)
• Encouraged systematic research on peer review
• Introduction of simple but mandatory checklist in editorial process:
4 criteria: randomization, blinding, sample size, and exclusions
that if properly reported might reduce the risk of bias
• Requirement for registration of randomized clinical trials
• Assessments of early positive experiences with data sharing
Among trials registered at ClinicalTrials.gov*:
33% completed trials & 57% terminated trials were not published
* Rennie D, Flanagin A. Three Decades of Peer Review Congresses.
JAMA 2018;319:350-53.

35. Role of policy makers
• Policy makers are interested and invest in peer review
• The EU initiated the project PEERE (www.peere.org)
• Multidisciplinary international project running from
2014 to 2018, collects quantitative & qualitative research
on how peer review is conducted, & seeks to make evidence-
based recommendations to improve the current system
PEERE: new frontiers of peer review

36. As long as humans are involved in judging
the scientific quality of a manuscript,
a completely bias-free system is unlikely

37. Conclusion
• Peer review process is far from perfect
• All of its systems entwined with advantages & challenges
• Double-blind review system is supported on equipoise
& fair-play principles
• There is little sound research on peer review, it is time to
perform large experimental systematic studies on various
peer review practices to improve quality & transparency

38. • Improve research
• Channel appropriately novel findings & observations
• Detect fraud when it occurs
Despite these limitations and concerns,
the peer-review process remain an invaluable tool to

39. Thank You