Systematic Review and
Meta-analysis
By:
Dr Pramoda N
MD Pharmacology

Overview
• Introduction
• Level of evidence
• Steps of Systematic Review
• Data analysis
• Publication bias
• Sensitivity analysis
• Limitations
• Summary
Systematic Review and Meta-analysis 4/2/2023 2

Introduction
• With increasing number of studies being published in the health
sciences, it is hard to keep up with the literature
• Primary care physicians need evidence for both clinical practice and
for public health decision making
• Reviews which combine and analyze multiple studies are helpful
• All the reviews are not systematic
4/2/2023
Systematic Review and Meta-analysis 3

Introduction
• Traditional narrative reviews
• Systematic review collects all possible studies related to a given
topic and design, and reviews and analyzes their results
• During the systematic review process, the quality of studies is
evaluated and a statistical meta-analysis of the study results is
conducted
• Systematic review and Meta-analysis have the highest level of
evidence in evidence-based medicine
4/2/2023
Systematic Review and Meta-analysis 4

Level of Evidence
4/2/2023
Systematic Review and Meta-analysis 5

Terminologies
• Systematic review is an objective, reproducible method to find
answers to a certain research question, by collecting all available
studies related to that question and reviewing and analyzing their
results.
• Meta-analysis uses statistical methods on estimates from two or
more different studies to form a pooled estimate.
4/2/2023
Systematic Review and Meta-analysis 6

Why meta-analysis?
Systematic Review and Meta-analysis 4/2/2023 7
Increase the power
To establish whether there is an effect
To understand the size of the effect and the certainty around it
To investigate if the effect is consistent across studies
To settle controversies arising from apparently conflicting studies or to generate new hypotheses

Steps of
Systematic
review
Systematic Review and Meta-analysis 4/2/2023 8
Formulating a research question
Protocol and registration
Defining inclusion and exclusion criteria
Literature search and study selection
Quality of evidence
Data Extraction
Analyzing data
Result presentation

Formulating research question
• First step involves defining the review question, forming
hypotheses and developing review title
• Define the Population, Intervention, Comparison, Outcome
(PICO) parameters
• It may be related to a major public health problem or a
controversial clinical situation
4/2/2023
Systematic Review and Meta-analysis 9

Protocol and Registration
• Prior registration of a detailed research plan is important
• Primary/secondary outcomes and methods are set in advance
• Many studies are registered with an organization like
PROSPERO.
• Registration number is recorded when reporting the study
4/2/2023
Systematic Review and Meta-analysis 10

Defining inclusion and exclusion criteria
• PICO
• Study design
• Publication status
• Language used
• Research period
4/2/2023
Systematic Review and Meta-analysis 11

Literature search and study selection
• Essential to perform a broad search with keywords
• Common bibliographic databases: Medline, Embase, Cochrane
Central Register of Controlled Trials (CENTRAL), Pubmed etc.,
• Identify both published as well as unpublished and ongoing
studies
• Remove duplicates
4/2/2023
Systematic Review and Meta-analysis 12

Literature search and study selection
• Select studies that meet inclusion and exclusion criteria based
on the abstracts
• Final selection based on full text
• Keep a log of all excluded studies with reasons
• Done independently by at least two investigators
• Essential to ensure reproducibility of literature selection
4/2/2023
Systematic Review and Meta-analysis 13

Quality of evidence
• Evaluating quality of evidence of studies helps determine the
strength of recommendation in the meta-analysis
• Grading of Recommendations, Assessment, Development and
Evaluations (GRADE) system
• 5-point Oxford Rating Scale (Jadad scale)
• The Cochrane Collaboration’s Tool for Assessing the Risk of
Bias
4/2/2023
Systematic Review and Meta-analysis 14

Quality of evidence
Systematic Review and Meta-analysis 4/2/2023 15
GRADE system
• Study limitations
• Inaccuracies
• Incompleteness of outcome data
• Indirectness of evidence
• Risk of publication bias

Quality of evidence
4/2/2023
Systematic Review and Meta-analysis 16
5-point Oxford Rating Scale (Jadad scale)
• Randomization – 2 points
• Blinding – 2 points
• An account of all patients – 1 point

Quality of evidence
4/2/2023
Systematic Review and Meta-analysis 17
The Cochrane Collaboration’s Tool for Assessing the Risk of Bias
• Sequence generation
• Allocation concealment
• Blinding
• Incomplete outcome data
• Selective reporting
• Other bias

Data Extraction
• Create and use a simple data extraction form or table
• Two different investigators extract data based on the
objectives and form of the study
• Differences in size and format of outcome variables
• If it is not possible to combine data – may be limited to
Systematic review
4/2/2023
Systematic Review and Meta-analysis 18

Data Analysis
• Aim of a meta-analysis is to derive a conclusion with increased
power
• Data are reviewed qualitatively and quantitatively
• Qualitative review - If data can not be combined, results of
individual studies are displayed
• Quantitative review - Meta-analysis – Calculating the weighted
pooled estimate for the interventions in at least two studies
4/2/2023
Systematic Review and Meta-analysis 19

Data Analysis
• Subgroup analysis - Should be planned in protocol
• The outcome of the meta-analysis is typically expressed using a
forest plot.
4/2/2023
Systematic Review and Meta-analysis 20

4/2/2023
Systematic Review and Meta-analysis 21

Data Analysis
To combine outcome variables:
• Dichotomous variables – Odds ratio, Risk ratio or risk
difference is used
• Continuous variables – Mean difference (MD) and Standardized
mean difference (SMD) is used
• Survival or time to event data - Hazard ratio is used
4/2/2023
Systematic Review and Meta-analysis 22

Data Analysis
• MD = Absolute difference between the mean value in two groups
• SMD =
Difference in mean outcome between groups
Standard deviation of outcome among participants
4/2/2023
Systematic Review and Meta-analysis 23

Data Analysis
• Risk ratio and risk difference can be used for RCTs, cohort
studies
• Odds ratio for case control or cross-sectional studies
• Number needed to treat (NNT) - Minimum number of
patients who need to be treated in the intervention group,
compared to the control group, for a given event to occur in
atleast one patient
4/2/2023
Systematic Review and Meta-analysis 24

Calculating NNT
Event occured Event not occured Sum
Intervention a b a + b
Control c d c + d
4/2/2023
Systematic Review and Meta-analysis 25
If x is the probability of event occurring in control group
And y is the probability of event occurring in intervention group, then
x = c/(c+d) and y = a/(a+b)
Absolute risk reduction (ARR) = x – y
NNT = 1/ARR

Types of models
In order to analyze effect size, two types of models can be used:
1. Fixed-effect model
2. Random-effect model
4/2/2023
Systematic Review and Meta-analysis 26

Fixed-effect model
• A fixed effect model assumes that the effect of treatment is the
same, and that variation between results in different studies is
due to random error
• Can be used when studies have same design and methodology, or
when the variability in results within a study is small
4/2/2023
Systematic Review and Meta-analysis 27

Random-effect model
• Random effect model assumes heterogeneity between the studies
being combined
• These models are used when the studies are assumed different, even
if a heterogeneity test does not show a significant result
• Weight does not decrease greatly for studies with a small number
of patients
4/2/2023
Systematic Review and Meta-analysis 28

4/2/2023
Systematic Review and Meta-analysis 29

Heterogeneity
• Three types of tests can be used:
Forest plot
Cochrane’s Q test(chi-squared)
Higgins 𝐼2
statistics
4/2/2023
Systematic Review and Meta-analysis 30

Publication bias
• Distortion of meta-analysis outcomes due to higher
likelihood of publication of statistically significant studies
rather than non-significant studies
• Most common type of reporting bias in meta-analysis
• To test publication bias – funnel plot
• To test statistically – Begg and Mazumdar’s rank correlation
test or Egger’s test can be used
4/2/2023
Systematic Review and Meta-analysis 31

Funnel plot
4/2/2023
Systematic Review and Meta-analysis 32
Publication bias absent publication bias present

Publication bias
• If Publication bias is
detected, the trim-
and-fill method can
be used to correct
the bias
Systematic Review and Meta-analysis 4/2/2023 33

Sensitivity analysis
• Used to determine how results of a systematic review or meta-
analysis change by fiddling with data.
• If changing makes little or no difference – conclusions are
robust
• If key findings disappear after change – conclusions need to
be expressed more cautiously
4/2/2023
Systematic Review and Meta-analysis 34

Result Presentation
• Analytical content and methods should be described in detail
• Flowchart is displayed with the literature search and selection
process
• A table is shown with characteristics of included studies
• A table included with information related to quality of evidence
• Data analysis – Forest plot and funnel plot
• If results use dichotomous data – NNT Values can be reported
4/2/2023
Systematic Review and Meta-analysis 35

Guidelines for reporting meta-analysis
• Quality of Reporting of Meta-analysis (QUORUM) statement
• Preferred Reporting Items for Systematic Reviews and Meta-
Analysis (PRISMA) statement
4/2/2023
Systematic Review and Meta-analysis 36

Result Presentation
• Review Manager (RevMan) software which is used for meta-
analysis, gives two types of P values
First P value for the z-test (most important)
Second P value is from Chi-squared test
4/2/2023
Systematic Review and Meta-analysis 37

4/2/2023
Systematic Review and Meta-analysis 38

Limitations of SRMA
• Quality depends on what was published in literature
• Takes longer time
• Can quickly be outdated – Needs to be updated
• There may not be enough research in the literature to analyze
4/2/2023
Systematic Review and Meta-analysis 39

Summary
• Systematic Reviews and Meta-analysis – Highest level of evidence
• Make the research literature easily accessible with increased accuracy
• Good research question and literature search are key initial steps
• Quality of studies included in the review have impact on quality of review
4/2/2023
Systematic Review and Meta-analysis 40

Summary
• Meta-analysis may not be possible in all Systematic reviews
• Results of meta-analysis are presented on forest plot with pooled estimate
from individual studies
• Publication bias is most common reporting bias in Meta-analysis
• Reporting of Systematic review and meta-analysis – PRISMA 2009 Checklist
4/2/2023
Systematic Review and Meta-analysis 41

References
• Tawfik, G. M., Dila, K. A. S., Mohamed, M. Y. F., Tam, D. N. H., Kien, N. D., Ahmed,
A. M., & Huy, N. T. (2019). A step by step guide for conducting a systematic
review and meta-analysis with simulation data. Tropical Medicine and
Health, 47(1), 46.
• Ahn, E., & Kang, H. (2018). Introduction to systematic review and meta-
analysis. Korean Journal of Anesthesiology, 71(2), 103–112.
• Gopalakrishnan, S., & Ganeshkumar, P. (2013). Systematic reviews and meta-
analysis: Understanding the best evidence in primary healthcare. Journal of
Family Medicine and Primary Care, 2(1), 9–14.
• Uman, L. S. (2011). Systematic reviews and meta-analyses. Journal de l’Academie
Canadienne de Psychiatrie de l’enfant et de l’adolescent [Journal of the Canadian
Academy of Child and Adolescent Psychiatry], 20(1), 57–59.
4/2/2023
Systematic Review and Meta-analysis 42

Thank you
4/2/2023
Systematic Review and Meta-analysis 43