This document provides an overview of meta-analysis. It defines meta-analysis as a quantitative approach to systematically combining results from previous studies to arrive at conclusions about the body of research. It discusses key aspects of planning and conducting a meta-analysis such as defining the research question, searching for relevant literature, determining study eligibility, extracting data, analyzing effect sizes, assessing heterogeneity, and addressing publication bias. Software for performing meta-analyses and specific effect sizes like risk ratio and odds ratio are also mentioned.

1. 張偉豪
三星統計服務有限公司 執行長
SEM 亞洲一哥
統計黑傑克
版次:20151225

2. 
If you can't explain it simply,
you don't understand it well
enough.
Albert Einstein

3. 
What is Meta-Analysis
Software of Meta-Analysis
How to plan a Meta-Analysis
RCT, Cohort study or Case study
Effect size
Risk ratio(RR) vs. Odds ratio(OR)
Fix effect vs. Random effect
Heterogeneity test
Publication bias
Reporting the results of a Meta-Analysis
Outline

4. 
Books

5. 
Meta-analysis is a quantitative
approach for systematically
combining results of previous
research to arrive at conclusions
about the body of research.
What is Meta-Analysis

6. 
Quantitative : numbers
Systematic : methodical
Combining: putting together (mean and
variance)
Previous research: what's already done
Conclusions: new knowledge
What is Meta-Analysis

7. 
When individual trials or studies’ sample sizes
are too small to give reliable answers.
When large trials or studies are impractical or
impossible
 Potentially lead to more timely introduction of
effective treatment
When there have been many trials or studies
showing small effects may be important.
Avoid institutional review board (IRB) censor.
7
Advantages of Use of Meta-Analysis

8. 
Hierarchy of evidence
Meta-Analysis
Systematic Review
Randomized Controlled Trial
Cohort studies
Case Control studies
Case Series/Case Reports
Animal research

9. 
Individual studies
 Collecting similarity studies from previous research.
Effect sizes (ES)
 Transform data (analysis results) into effect size to
reflect the magnitude of treatment effect or the
strength of a relationship between two variables.
Precision
 The effect size for each study is bounded by as
confidence interval (CI), reflect the precision of effect
size.
How a Meta-Analysis work

10. 
Study weight
 Ideal studies (sample size are larger) are assigned
relatively high weight.
P-value
 A p-value for a test of the null hypothesis.
 If p<0.05 reject null hypothesis.
The summary effect
 Summary the effect size from all studies, including
mean ES (fix effect), CI, weight, p-value, ES
heterogeneity, random effect, publication bias etc..
How a Meta-Analysis work

11. 
CMA is able to accept data in more
than 100 formats and allows the user
to mix and match formats in the same
analysis.
CMA is able to perform fixed-effect
and random-effects analyses. They all
report the key statistics, such as the
summary effect and confidence
intervals, measures of heterogeneity
(T2, Q, I2)
CMA allow the researcher to automate
the process, performing the analysis
repeatedly and removing a different
study on each pass.
Why use Comprehensive Meta-
Analysis (CMA)

12. 
Why use Comprehensive Meta-Analysis
(CMA)
CMA allows the user
to define a hierarchical
structure and then
offers the user a set of
options including the
option to create a
synthetic variable
based on some (or all)
the outcomes, or to
work with each
outcome separately.
CMA offer a full set of
tools to assess
publication bias.
CMA support 50
formats for data entry,
all of the basic
computational
options, and high-
resolution forest
plots.

13. 
Define the
Research
Question
Perform the
literature
search
Determine
eligibility of
studies
Extract the
data from
studies
Analyze the
data in the
study
statistically
Examine
heterogeneity
Assess
publication
bias
Interpret and
Report the
results
Steps in a meta-analysis

14. Eight Steps of Meta Analysis
1. Define the Research Question
2. Perform the literature search
3. Determine eligibility of studies
 Inclusion: which ones to keep
 Exclusion: which ones to throw out
4. Extract the data from studies
5. Analyze the data in the study statistically
6. Examine heterogeneity
7. Assess publication bias
8. Interpret and Report the results
How to plan a Meta-Analysis

15. 
In patients with coronary artery disease (CAD)
does vitamin E supplementation decrease the
risk of death?
Patients digest Carotenoids will decrease the
chance of lung cancer happen.
Define the Research Question
Define the
Research
Question

16. 
Potentially relevant
references identified after
liberal screening of the
electronic search (n=#)
Excluded by
Title/Abstract (n=#)
List the reasons
Articles retrieved for more
detailed evaluation (n=#)
Articles excluded after
evaluation of full text
(n=#) List the reasons
Relevant studies included in
the meta-analysis (n=#)
Flow Diagram of Study
Selection Process

17. 
Be methodical: plan first
List of popular databases to search
Pubmed/Medline/Embase
List every possible database you may search.
Other strategies you may adopt
Hand search (go to the library...)
Personal references, and emails
web, eg. Google scholar
(http://scholar.google.com)
Identify your studies
Perform the
literature
search

18. 
Let's say we want to know that passive smoking
really cause lung cancer.
How should we set up a search strategy?
 What is the key words?
 “Smoking” or/and “lung cancer”
 Passive/Second hand smoking
 Active smoking
 Air pollution
 Lung disease
Search key word

19. 
“passive smoking” OR “second hand
smoking”[text word] OR lung cancer
produces ALL articles that contain EITHER
smoking OR lung cancer to get a lot of
articles.
“Passive smoking” AND “lung cancer” will
capture only those subsets that have
BOTH smoking AND lung cancer reduce
the articles.
The Search

20. 
Cannot include all studies
Keep the ones with
 high levels of evidence
 good quality
Usually, MA done with RCTs
Case series, and case reports definitely out
Selection problems are major problems
Keep some, throw out others
Determine
eligibility of
studies

21. 
Are the studies similar enough to combine?
Can I combine studies with different designs?
 Experiential VS. Observational
Studies that used independent groups, paired
groups, clustered groups
Can I combine studies that report results in
different ways?
How many studies are enough to carry out a
meta-analysis?
When Does it Make Sense to
Perform a Meta-Analysis?

22. 
Randomized Controlled Trials (RCTs)
• The cases who was random select from population
• Belong to experimental study
• Exposure didn’t naturally
• Blind randomized trial
RCT, Cohort study or Case study

23. 
Cohort Study is any group of people who are
linked in some way and followed over time.
 Belong to observational study
 Expose naturally in nature world
 Prospective Cohort study
 Retrospective Cohort study
 Time Series Study
Case Control
 examine associations between disease/disorder/health
issue and one or more risk factors
RCT, Cohort study or Case study

24. 
Question: Will smoke behavior cause lung
cancer?
 Prospective Cohort study
 Causality research
 Find multiple consequence
 Retrospective Cohort study
 Find multiple causes may cause diseases
 Outcome is determined before exposure status
 No need huge sample size
Cohort study

25. 
Researchers use existing records to identify
people with a certain health problem (“cases”)
and a similar group without the problem
(“controls”).
 Similar retrospective Cohort study
 Example: To learn whether a certain drug causes birth
defects, one might collect data about children with
defects (cases) and about those without defects
(controls).
 The data are compared to see whether cases are
more likely than controls to have mothers who took
the drug during pregnancy.
Case control study

26. 
Create a spreadsheet (Excel, or OpenOffice Calc)
For each study, create the following columns:
 name of the study
 name of the author, year published
 number of participants who received intervention
 number of participants who were in control
 number who developed outcomes in intervention
 number who developed outcomes in control
How to Abstract Data
Extract the
data from
studies

27. Spreadsheet Data for Strepto Study
We created seven columns
trial: trial identity code
trialname: name of trial
year: year of the study
pop1: study population
deaths1: deaths in study
pop0: control population
deaths0: deaths in control
There are 22 studies to do our meta analysis

28. 
Data entry

29. 
The properties of effect size in a
meta-analysis
be comparable across studies (standardization)
represent magnitude & direction of the relationship
be independent of sample size
Effect size

30. 
The ES makes meta-analysis possible
The ES encodes the selected research findings
on a numeric scale
There are many different types of ES measures,
each suited to different research situations
Each ES type may also have multiple methods of
computation
Effect size (ES)

31. Standardized mean difference
 Group contrast research
 Cohen’s d = 02, 0.5, and 0.8 as a small, medium, and large
effect size
 Output is continuous.
Odds-ratio
 Group contrast research
 OR = 1.68, 3.47, and 6.71 as a small, medium, and large
effect size
 Output is dichotomous.
Correlation coefficient
 Association between variables research
31
Different Types of Effect Sizes

32. 
Odds definition
 The probability of event divided by the probability of
the alternative.
 Odds = p/1-p
𝑶𝑹 =
𝑶𝒅𝒅𝒔 𝒐𝒇𝒆𝒙𝒑𝒐𝒔𝒖𝒓𝒆 𝒊𝒏 𝒕𝒉𝒐𝒔𝒆 𝒘𝒊𝒕𝒉 𝒅𝒊𝒔𝒆𝒂𝒔𝒆
𝑶𝒅𝒅𝒔 𝒐𝒇𝒆𝒙𝒑𝒐𝒔𝒖𝒓𝒆 𝒊𝒏 𝒕𝒉𝒐𝒔𝒆𝒘𝒊𝒕𝒉𝒐𝒖𝒕 𝒅𝒊𝒔𝒆𝒂𝒔𝒆
Interpretation
 OR>1 Increase frequency of exposure among cases
 OR=1 No change in frequency of exposure
 OR<1 Decrease frequency of exposure
An OR about 2 is usually important
Odds ratio(OR)

33. 
Definition of RR
 The proportion experiencing the event in one group
divided by the proportion experiencing it in the other.
 RR = p1/p2
𝑹𝑹 =
𝑰𝒏𝒄𝒊𝒅 𝒐𝒇 𝒐𝒖𝒕𝒄𝒐𝒎𝒆 𝒘𝒊𝒕𝒉 𝒆𝒙𝒑𝒐𝒔𝒖𝒓𝒆
𝑰𝒏𝒄𝒊𝒅 𝒐𝒇 𝒐𝒖𝒕𝒄𝒐𝒎 𝒘𝒊𝒕𝒉𝒐𝒖𝒕 𝒆𝒙𝒑𝒐𝒔𝒖𝒓𝒆
RR is suitable Cohort studies
Interpretation
 RR>1 Increase risk of outcome
 RR=1 No risk of outcome
 RR<1 Reduce risk of outcome
Risk ratio(RR)

34. 
Fix effect
 Assumes that all studies are estimating the same true
effect
 Variability only from sampling of people within each
study
 Precision depends mainly on study size
Fix vs. Random effect

35. 
Random effect
 Studies allowed to have different underlying or true
effects
 Allows variation between studies as well as within
studies
Fix vs. Random effect

36. 
Random effects generally yield larger variances
and CI
 Why? Incorporate
If heterogeneity between studies is large
between variance, will dominate the weights
and all studies will be weighted more equally
Model weight for large studies less in random vs.
fixed effects model
Fix vs. Random effect

37. 
Statistical test for heterogeneity
Visual inspection/Graphical approach
 Forest plot
Meta-regression
 Unit of regression: study
 Dependent variable: study-specific effect estimate
 Independent variables: study-specific characteristics
(e.g., study design, geographic location, length of
follow-up)
37
Examining Heterogeneity
Examine
heterogeneity

38. 
Different study designs
Different incidence rates among unexposed
Different length of follow-up
Different distributions of effect modifiers
Different statistical methods/models used
Different sources of bias
Study quality
Sources of Between Study
Heterogeneity

39. 
39
Examining Forest Plot for
Heterogeneity

40. 
The I2 statistic describes the percentage of variation
across studies that is due to heterogeneity rather
than chance. .
I2 statistic value is a standardized value.
I2 statistic (between variance/total variance)
1. 0% ~ 40%: heterogeneity might not be important;
2. 30% ~ 60%: may represent moderate heterogeneity;
3. 50% ~ 90%: may represent substantial heterogeneity;
4. 75% ~ 100%: considerable heterogeneity.
Heterogeneity test

41. 
In traditional (fixed-effects) meta-analysis
heterogeneity test using the Q statistic.
 The test has low power, so you use p<0.10 rather than
p<0.05.
 If p<0.10, you exclude "outlier" studies and re-test,
until p>0.10.
 When p>0.10, you declare the effect homogeneous.
Heterogeneity test

42. 
Strategies for addressing
heterogeneity
Check again that the data are correct
Do not do a meta-analysis
Explore heterogeneity (subgroup analysis,
meta-regression)
Ignore heterogeneity (there is no an
intervention effect but a distribution of
intervention effects)
Perform a random-effects meta-analysis
(when heterogeneity cannot be explained)
Change the effect measure (different scales
in different studies)
Exclude studies (outlying studies)

43. 
Sensitivity analysis
Sensitivity analysis have been used to
examine the effects of studies identified
as being aberrant concerning conduct or
result, or being highly influential in the
analysis.
One study removed meta-analysis
Cumulative analysis

44. how the results would change if one study (or a
set of studies) was removed from the analysis.
One study removed meta-analysis

45. 
A cumulative meta-analysis is performed first
with one study, then with two studies, and so on,
until all relevant studies have been included in
the analysis.
A cumulative analysis entering the larger studies
at the top and adding the smaller studies at the
bottom, sorted by sample size or precision.
A benefit of the cumulative analysis is that it
displays not only if there is a shift in effect size,
but also the magnitude of the shift.
Cumulative analysis

46. 
What is Meta-Analysis bias?
Can bias the results of a meta-analysis toward a
positive finding
Can evaluate publication bias graphically (funnel
plot) or through statistical analysis
Test of Publication Bias
Assess
publication bias

47. 
Outcome reporting bias
 Significant outcomes are more likely to be reported
than non-significant outcomes.
Should unpublished data be included in systemic
review?
 Pre-specified inclusion (quality) criteria are
recommended.
Database Bias
 No single database is likely to contain all published
studies on a given subject.”
Where Can Publication Bias Occur?

48. 
Publication Bias
 selective publication of articles that show positive
treatment of effects and statistical significance.
English-language (duplication) bias
 Studies with statistically significant results are more likely
to be published in English
Citation bias
 occurs when studies with significant or positive results are
referenced in other publications, compared with studies
with inconclusive or negative findings
Meta-Analysis bias

49. 
Funnel plot
Rosenthal’s Fail-safe N
Orwin’s Fail-safe N
Duval and Tweedie’s Trim & Fill
rank correlation (P>0.05)
Regression
Methods for addressing
publication bias

50. 
Funnel plot has several caveats:
1. funnel plot may yield a very different picture
depending on the index used in the analysis
(risk difference versus risk ratio).
2. Funnel plot makes sense only if there is a
reasonable amount of dispersion in the sample
sizes and a reasonable number of studies.
3. even when these criteria are met, the tests
tend to have lower power.
Funnel plot

51. 
The absence of a significant correlation or
regression cannot be taken as evidence of
symmetry.
To solve these problems, we use
Rosenthal’s Fail-safe N
Orwin’s Fail-safe N
Duval and Tweedie’s Trim and Fill
Funnel plot

52. 
 What is our best estimate of the unbiased effect
size?
 Trim and fill procedure will tell you the answer, the
method separate into trim and fill two steps.
Trim & fill

53. 
 Trim first
 remove the most extreme small studies from the
positive side of the funnel plot, re-computing the
effect size at each iteration until the funnel plot is
symmetric about the (new) effect size.
 yields the adjusted effect size
(unbiased summate ES).
Fill follow
 adds the original studies back into the analysis, and
imputes a mirror image for each.
 to correct the ES variance.
Trim and Fill procedure

54. 
 The fail-safe N (Rosenthal, 1991) determines the
number of studies with an effect size of zero
needed to lower the observed effect size to a
specified (criterion) level.
The fail-safe N actually compute how many missing
studies we would need to retrieve and incorporate
in the analysis before the p-value became
nonsignificant..
Rosenthal’s Fail-safe N
(File drawer analysis)

55. 
the Fail-safe N is 38, suggesting that there would
need to be nearly 40 studies with a mean risk ratio
of 1.0 added to the analysis, the research will
become statistically nonsignificant.
Rosenthal’s Fail-safe N

56. 
Orwin’s method allows the researcher to
determine how many missing studies would
bring the overall effect to a specified level other
than zero.
it allows the researcher to specify the mean
effect in the missing studies as some value other
than zero.
Orwin’s Fail-safe N

57. 
Begg and Mazumdar
rank correlation

58. 
Is there evidence of bias?
Egger’s regression

59. 
Combine data to arrive at a summary,
3 measures
 Effect Size (Odds Ratio or Risk Ratio or Correlations)
 Variance with 95% Confidence Interval
 Test of heterogeneity
Two Graphs
 Forest Plot
 Funnel Plot
Examine why the studies are heterogeneous
Examine publication bias.
Reporting the results
Interpret and
Report the
results

60. 
Meta-Analysis check list

61. 
Are the studies similar enough to combine?
 There is no restriction on the similarity of studies
Based on the types of participants, interventions, or
exposures.
Can I combine studies with different designs?
 Randomized trials versus observational studies
 Studies that used independent groups, paired groups,
clustered groups
 Can I combine studies that report results in different
ways?
When Does it Make Sense to
Performa Meta-Analysis?

62. 
How many studies are enough to carry out a
meta-analysis?
Fix effect model
 At least two studies, since a summary based on two
or more studies yields a more precise estimate of the
true effect than either study alone.
Random effect model
When Does it Make Sense to
Performa Meta-Analysis?

63. 
One number cannot summarize a research field
The file drawer problem invalidates meta-
analysis
Mixing apples and oranges
Garbage in, garbage out
Important studies are ignored
Meta-analysis can disagree with randomized
trials
Meta-analyses are performed poorly
Criticisms of Meta-Analysis

64. 