2. Outline
I. What is systematic review
II. Why to do systematic review
III. When to do systematic review
IV. Model of a Cochrane systematic
review
V. Steps to do a systematic review
VI. Challenges of systematic reviews
3. I. What is a systematic
review?
A review of a clearly formulated
question that
uses systematic and explicit methods to
1. identify, select and critically appraise
relevant research
2. collect and analyse data from the
studies that are included in the review
(Cochrane Reviewers’ Handbook 4.1.5)
5. QUESTION Broad Focused
SOURCES/ Usually unspecified Comprehensive;
SEARCH Possibly biased explicit
SELECTION Unspecified; biased? Criterion-based;
uniformly applied
APPRAISAL Variable Rigorous
SYNTHESIS Usually qualitative Quantitative
INFERENCE Sometimes Evidence-based
evidence-based
NARRATIVE SYSTEMATIC
Cook, D. J. et. al. Ann Intern Med 1997;126:376-380
6. Level of evidence
• I–1 Systematic reviews.
I–2 One or more large double-
blind RCT.
• II–1 One or more well-
conducted cohort studies.
II–2 One or more well-
conducted case-control
studies.
II–3 uncontrolled experiment.
• III Expert opinion.
• IV Personal experience
7. Why on the Top
• Rigorous methodology
• Peer reviewed
• Relatively large sample size
• Ensures the highest quality evidence
8. II. Why do we need it?
• Too much trials
25000 biomedical journals in print
8000 articles published per day
• All studies not equally well designed or interpreted
So, we need a study of studies
• To summarize evidence from studies that address a
specific clinical question.
• To explain differences among studies on the same
question
• To limit bias (rigorous methodology & clear reporting)
9. Example
• Protocols in neurology units - 80% still
recommend bed rest after LP
• Systematic review of 10 trials of bed rest after
spinal puncture
– no change in headache with bed rest
– Increase in back pain
Serpell M, BMJ 1998;316:1709–10
10. • Many single trials had relatively low power
• Avoid Type II error: Investigators did not detect a
difference when a difference actually exists
• This is not surprising as the power to detect a
difference will have been increased by the increase
in the sample size
• Systematic reviews help us to avoid the personal
bias inherent in traditional reviews and expert
opinion
• Results from systematic reviews are the
cornerstone for developing practice guidelines
11. III. When can you do meta-
analysis?
• When more than one study has
estimated an effect
• When there are no differences in the
study characteristics that are likely to
substantially affect outcome
• When the outcome and treatment effect
have been measured in similar ways
• When the data are available
12. When not to do a meta-analysis?
• ‘garbage in – garbage out’
– a meta-analysis is only as good as the studies in
it
– meta-analysis in presence of serious publication
and/or reporting biases may produce an
inappropriate summary
14. Cochrane: Prof. Archie Cochrane
CBE, FRCP, FFCM (1909-1988)
1960-74: Director, MRC Epidemiology
Research Unit
1972: Publication by Nuffield Provincial
Hospital Trust of his book
“Effectiveness and Efficiency :
Random Reflections on Health
Services”
“It is surely a great criticism of our
profession that we have not organised a
critical summary, by specialty or
subspecialty, adapted periodically, of all
relevant randomised controlled trials.”
15. The Cochrane Collaboration
• International collaboration
• Prepares, maintains, and disseminates
systematic reviews
• Diverse internal structure (Review Groups,
Centres, Fields, Methods Groups, the
Consumer Network)
Cochrane Library
• The current resource with the highest methodological
rigor
• $235/year or abstracts only
• www.cochrane.org
17. The logo
represents meta-analysis of 7 trials of IM
corticosteroids given to mothers for
foetal maturation in preterm infants
[Chalmers et al].
18. Abstract
Background
Objectives
Criteria for considering studies for this review
Types of participants
Types of intervention
Types of outcome measures
Types of studies
Search strategy for identification of studies
Methods of the review
Description of the studies
Methodological qualities of included studies
Results
Discussion
Conclusions
Implications for practice
Implications for research
Internal sources of support to the review
External sources of support to the review
Potential conflict of interest
Acknowledgements
Contribution of Reviewer(s)
Synopsis
Characteristics of included studies
Table 01 results
References to studies included in this review
Additional references
Typical Systematic Review “Skeleton”
19. V. Steps to do a systematic review
• Well-Formulated Question
• Efficient Search Strategies
• Review Abstracts to Determine Eligibility
• Apply Strict Inclusion/Exclusion Criteria
• Extract the Data
• Perform the Required Analyses (Meta-
analysis)
• Interpret the Results
• Determine Implications for Health Care Policy
and Practice
20. Methodology
• At least 3 reviewers
• Detailed description of :
Trial design characteristics
Why included / excluded
Quality of included studies in details
Source of articles
• Electronic databases
• Bibliography of selected articles
• Hand searching Journals
• “Gray” Literature
• Key Informants
• Web Searching
21. VI. Challenges for systematic
reviews
• Evidence into practice
• Many interventions reviewed cannot be
implemented in resource-poor situations
• Most interventions reviewed so far don’t
reflect developing world priorities
• Very few studies that have been conducted in
a developing country
• Most developing country research that is
found is excluded on quality grounds
