1. Meta Analysis Statistical analysis for Systematic Reviews of Randomised Clinical Trials  Contents

2. Contents - Example - History – Fisher/Cochran, Peto, Cochrane, EBM - Cochrane centre and Cochrane collaboration. - Systematic reviews - Statistical methodology - combining results - testing heterogeneity - comparability and quality of studies - publication bias - Examples and Stata analysis - Current issues – x-over trials, non RCTs, epidemiological studies - Conclusions   blockers eg

3. Example: Reports of several clinical trials of the effect of  blockers on survival after Myocardial Infarction are available (Yusuf et al looked at 65 trials of different types). The main outcome variable is live/dead status at 1 month after treatment started. The main test statistic is the Odds Ratio of surviving (treatment .v. placebo). The statistical problems are: - How to combine the ORs and obtain an overall OR with CLs - How to assess and allow for the ‘true’ ORs varying with study ( Heterogeneity ) - How to assess and allow for the quality and comparability of the studies - How to assess whether these studies can be assumed a random sample of all such studies i.e. are some studies with different (?non-significant) results not being published – Publication Bias  History 1

4. History 2: Origins In the 1970s Richard Peto and others, because clinical trials of treatments for heart disease were often small and thus of low power, performed and published a number of systematic reviews of such trials. Peto identified or devised appropriate methods for combining test statistics such as odds ratios, relative risks and differences between proportions and means from different studies. In essence the problem is the same as that of multi-centre trials. But variation in patient types, treatment regimes, outcome measures, study design and general trial procedures is much greater between unrelated studies than between centres following the same trial protocol. This means that the aggregation of results has to be done very carefully and interpreted equally cautiously.  History 3

5. History 6: The Cochrane Collaboration In 1992 Iain Chalmers at the Oxford Perinatal Research Unit and others set up the Cochrane Centre in Oxford to encourage and document systematic reviews of the effectiveness of Health interventions. It was named after a distinguished epidemiologist Archie Cochrane who in 1979 had written: "It is surely a great criticism of our profession (Medicine) that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized controlled trials.” http://hiru.mcmaster.ca/cochrane/cochrane/archieco.htm In October 1993 - at what was to become the first in a series of annual Cochrane Colloquia - 77 people from eleven countries co-founded 'The Cochrane Collaboration’ to encourage and document systematic reviews world-wide. This happened in parallel with the ‘Evidence based medicine’ movement started by Sackett and others.  Systematic reviews

6. Proposers need to specify: - the condition or disease - the treatment comparisons of interest - the types of study to be reviewed - the strategy for searching the literature and elsewhere - the outcome variables and planned analysis. The Cochrane Collaboration publish guidelines entitled “ How to conduct a Cochrane review” Systematic Reviews  Stroke eg

7. Results from 9 studies comparing Length of Stay (LOS) in hospital of Stroke patients receiving Specialist and Routine care Specialist care Routine management Mean Mean N LOS SD N LOS SD 155 55 47 156 75 64 31 27 7 32 29 4 75 64 17 71 119 29 18 66 20 18 137 48 8 14 8 13 18 11 57 19 7 52 18 4 34 52 45 33 41 34 110 21 16 183 31 27 60 30 27 52 23 20  paired plot

8. Stroke studies paired mean LOS Mostly shorter LOS with specialist care, but some evidence of heterogeneity  Formulae Paired t test structure

9. Combining Results The primary outcome in this study is the mean length of stay (LOS). So for study i we have: Study Treatment 1 Treatment 2 Difference i No. Mean s.d. No. Mean s.d. 1 N 11 y 11 s 11 N 12 y 12 s 12 . .. .. .. .. .. .. i N i1 y i1 s i1 N i2 y i2 s i2 y i = y i1 - y i2 . .. .. .. .. .. .. k N k1 y k1 s k1 N k2 y k2 s k2 pooled variance s pi 2 = (( N i1 -1) s i1 2 + ( N i2 -1) s i1 2 )/( N i1 + N i2 -2) var ( y i ) = s i 2 = s pi 2 (1/ N i1 + 1/ N i2 ) W i = 1/ s i 2 Combined estimate =  W i y i /  W i with variance = 1 /  W i  General Formulae

10. Formulae for combining test statistics Normand, S-L. T (1999) Statist. Med 18, 321-359 Fixed-effects model: ~  Stroke results

11. Results from 9 studies comparing Length of Stay (LOS) in hospital of Stroke patients receiving Specialist and Routine care Difference 95% CLs Weights W i . bet. Means Low High Fixed Random -20 -32.5 -7.5 0.0246 0.0036 -2 -4.8 0.8 0.4886 0.0042 -55 -62.7 -47.3 0.0654 0.0040 -71 -95.0 -47.0 0.0067 0.0026 -4 -12.8 4.8 0.0495 0.0039 1 -1.2 3.2 0.8173 0.0043 11 -8.1 30.1 0.0105 0.0030 -10 -15.6 -4.4 0.1245 0.0041 7 -1.9 15.9 0.0483 0.0039 Combined estimate MD w =  W i y i /  W i = -3.5 with variance = 1 /  W i = 0.6115 ............. Q =  W i ( y i - MD w ) 2 = 241.1 ~  2 8  Forest plot

12. The overall weighted mean differences are : Mean 95% CLs Fixed effects: -3.5 -5.0 -2.0 Random effects: -14.2 -24.8 -3.5 Test of heterogeneity: Q = 241.1 Treated as  2 with 8 df p = 0.0000  Funnel plot Individual and combined results

13. Assessing publication bias  Meta analysis in RevMan

14. Meta analysis in Stata can be performed with the Metan command written by: Michael J Bradburn , Jonathan J Deeks , Douglas G Altman Institute of Health Sciences, Oxford Metan help etc...  Lidocaine eg

15.  Results Lidocaine treatment for MI v Placebo S-L T Normand Statist. Med (1999) 18, 321-359 Meta Analysis in RevMan

16.  RevMan analysis of GBS data

17.  RevMan forest plot

18. Review Manager (RevMan) from the Cochrane Collaboration http://www.cochrane.org/software/revman.htm  New facilities in RevMan

21. Only need the first two columns… Difference 95% CLs Weights W i bet. Means Var(diff) Low High Fixed Random -20 40.6 -32.5 -7.5 0.0246 0.0036 -2 2.1 -4.8 0.8 0.4886 0.0042 -55 15.3 -62.7 -47.3 0.0654 0.0040 -71 150.2 -95.0 -47.0 0.0067 0.0026 -4 20.2 -12.8 4.8 0.0495 0.0039 1 1.2 -1.2 3.2 0.8173 0.0043 11 95.4 -8.1 30.1 0.0105 0.0030 -10 8.0 -15.6 -4.4 0.1245 0.0041 7 20.7 -1.9 15.9 0.0483 0.0039 C ombined estimate MD w =  W i y i /  W i = -3.5 -14.1 with variance = 1 /  W i = 0.6115 (-5.0,-2.0) (-24.5,-3.8) Q =  W i ( y i - MD w ) 2 = 241.1 ~  2 8 I 2 = 100*(Q – df)/Q = 96.7%  forest plot

22. Trials in same order as table – GIV needs just 2 values for each trial  Compare with previous

23. The overall weighted mean differences are : Mean 95% CLs Fixed effects: -3.5 -5.0 -2.0 Random effects: -14.2 -24.8 -3.5 Test of heterogeneity: Q = 241.1 Treated as  2 with 8 df p = 0.0000  Heterogeneity I 2 Individual and combined results

24. 2. The I 2 measure of heterogeneity(1) Heterogeneity – Systematic or Random Systematic heterogeneity may arise because studies differ for a specific reason e.g. quality of study, biases or errors, type of patient, other non-random factors … usually implies that studies should be omitted or analysed separately; Random heterogeneity between studies in the measures of the relative merits of two treatments is generally taken to represent differences that would occur in routine use of the treatments in similar institutions in the future. It is usually assumed to indicate that the combined estimate should be obtained using a random effects approach to take between study variation into account in the calculation of the combined estimate.  Stroke trials

25. Trials in same order as table – GIV needs just 2 values for each trial  outliers moved

26. All mean differences about –15 … I 2 = 24.6%

27. Areas - Cross - over trials (treated as 2 parallel groups is safe) - non randomised trials; - observational clinical studies; - epidemiological studies. Methodology - combining medians - linear model and survival data analysis results (GIV) - linear modelling approaches to combining study results; (stratifying by study quality, patient and treatment variations etc) - Bayesian methods - Others.... Current Issues  Conclusions

28. Conclusions - Meta analysis of epidemiological studies is know to be fraught with many more problems than arise in the process of combining randomised clinical trials -- extra care is needed; - must be aware that study results are data like any other and the problems are not different in principle to those faced in any statistical analysis; - raw data from the studies is always better than summarised results; - those doing a review should consult the Cochrane Guidelines.They need an exhaustive search strategy and careful classification of studies; - The analysis needs to be cautious, although the particular method may not be crucial, and the form and nature of any heterogeneity found needs to be considered carefully in context and taken into account in the analysis.  References

29. Reference List 1. Abrams K, Sanso B. Approximate Bayesian Inference for Random Effects Meta-analysis. Stats Med 1998; 17:2 :201-218. 2. Aitkin M. Meta-analysis by Random Effect Modelling in Generalized Linear Models. Stats Med 1999; 18:17/18 :2343-2353. 3. Austin H, Perkins LL, Martin DO. Estimating a Relative Risk Across Sparse Case-control and Follow-up Studies: A Method for Meta-analysis. Stats Med 1997; 16:9 :1005-1016. 4. Begg CB, Pilote L. A model for incorporating historical controls into a meta-analysis. Biometrics 1991; 47, No 3 :899-906. 5. Berkey CS, Hoaglin DC, Mosteller F, Colditz GA. A Random-effects Regression Model for Meta-analysis. Stats Med 1995; 14:4 :395-412. 6. Berkey CS, Hoaglin DC, Antozak-Bouckoms A, Mosteller F, Colditz GA. Meta-analysis of Multiple Outcomes by Regression with Random Effects. Stats Med 1998; 17:22 :2537-2550. 7. Berlin JA, Longnecker MP, Greenland S. Meta-analysis of Epidemiologic Dose-Response Data. Epidemiology 1993; 4 :218-228. 8. Berry SM. Understanding and Testing for Heterogeneity Across 2 x 2 Tables: Application to Meta-Analysis. Stats Med 1998; 17:20 :2353-2370. 9. Boutitie F, Gueyffier F, Pocock SJ, Boissel JP. Assessing Treatment-Time Interaction in Clinical Trials with Time to Event Data: A Meta-analysis of Hypertension Trials. Stats Med 1998; 17:24 :2883-2904. 10. Brand R, Kragt H. Importance of Trends in the Interpretation of an Overall Odds Ratio in the Meta-analysis of Clinical Trials. Stats Med 1993; 11 No 16 :2077-2082. 11. Brumback BA, Holmes LB, Ryan LM. Adverse Effects of Chorionic Villus Sampling: A Meta-analysis. Stats Med 1999; 18:16 :2163-2176. 12. Buyse M, Molenberghs G, Burzykowski T, Renard D, Geys H. The validation of surrogate endpoints in meta-analysis of randomized experiments. Biostatistics 2000; 1:1 :49-68. 13. Cole BF, Gelber RD, Goldhirsch A. A Quality-adjusted Survival Meta-analysis of Adjuvant Chemotherapy for Premenopausal Breast Cancer. Stats Med 1995; 14:16 :1771-1784. 14. Cooper H, Hedges LV. The Handbook of Research Synthesis . Sage Publications, 1994; 15. Copas J. What works?: selectivity models and meta-analysis. JRSS Series A 1999; 162:1 :95-110. 16. Dear KBG. Iterative generalized least squares for meta-analysis of survival data at multiple times. Biometrics 1994; 50:4 :989-1003. 17. Dersimonian R, Laird N. Meta-Analysis in Clinical Trials. Controlled Clinical Trials 1986; 7 :177-188. 18. DuMouchel WH. Meta-Analysis. Stats Med 1995; 14:5/6/7 :679-686. 19. Duval S, Tweedie R. Trim and Fill: A Simple Funnel-Plot-Based Method of Testing and Adjusting for Publication Bias in Meta-Analysis. Biometrics 2000; 56:2 :455-463. 20. Engels EA, Schmid CH, Terrin N, Olkin I, Lau J. Heterogeneity and statistical significance in meta-analysis: an empirical study of 125 meta-analyses. Stats Med 2000; 19:13 :1707-1728. 21. Fay MP, Graubard BI, Freedman LS, Midthune DN. Conditional logistic regression with sandwich estimators: application to a meta-analysis. Biometrics 1998; 54:1 :195-208. 22. Follman DA, Proschan MA. Valid Inference in Random Effects Meta-Analysis. Biometrics 1999; 55:3 :732-737. 23. Frost C, Clarke R, Beacon H. Use of Hierarchical Models for Meta-analysis: Experience in the Metabolic Ward Studies of Diet and Blood Cholesterol. Stats Med 1999; 18:13 :1657-1676. 24. Geller NL, Proschan M. Meta-analysis of clinical trials; a consumers guide. J Biopharmaceutical Stats 1996; 6 :377-394. 25. Goldstein H, Yang M, Omar R, Turner R, Thompson S. Meta-analysis using multilevel models with an application to the study of class size effects. JRSS Series C 2000; 49:3 :399-412. 26. Greenland S. Quantitative Methods in the Review of Epidemiologic Literature. Epidemiologic Reviews 1987; 9 :1-30. 27. Hardy RJ, Thompson SG. A Likelihood approach to Meta-analysis with Random Effects. Stats Med 1996; 15:6 :619-630.

30. Reference List 28. Hardy RJ, Thompson SG. Detecting and Describing Heterogeneity in Meta-analysis. Stats Med 1998; 17:8 :841-856. 29. Hedges, Olkin. Statistical Methods for Meta-Analysis . Boston: Academic Press, 1985; 30. Hutton JL, Williamson PR. Bias in meta-analysis due to outcome variable selection within studies. JRSS Series C 2000; 49:3 :359-370. 31. Larose DT, Dey DK. Grouped Random Effects Models for Bayesian Meta-analysis. Stats Med 1997; 16:16 :1817-1830. 32. Li Z. A Multiplicative Random Effects Model for Meta-Analysis with Application to Estimation of Admixture Component. Biometrics 1995; 51:3 :864-873. 33. MacMahon S, Peto R, Cutler et al. Blood pressure, stroke and coronary heart disease, part 1, prolonged differences in blood pressure: prospective observational studies corrected for the regression dilution bias. Lancet 1990; 335 :765-774. 34. Mathew T, Nordstrom K. On the Equivalence of Meta-Analysis Using Literature and Using Individual Patient Data. Biometrics 1999; 55:4 :1221-1223. 35. Mosteller F, Colditz GA. Understanding Research Synthesis (Meta-Analysis). Annu.Rev.Public Health 1996; 17 :1-23. 36. Normand SLT. Meta-analysis: Formulating, Evaluation, Combining, and Reporting. Stats Med 1999; 18:3 :321-360. 37. Olkin I. Meta-Analysis: Reconciling the Results of Independent Studies. Stats Med 1995; 14:5/6/7 :457-472. 38. Olkin I, Sampson A. Comparison of meta-analysis versus analysis of variance of individual patient data. Biometrics 1998; 54:1 :317-322. 39. Pettiti D. Meta-analysis, Decision Analysis and Cost-effectiveness Analysis . Oxford: OUP, 1994; 40. Platt RW, Leroux BG, Breslow N. Generalized Linear Mixed Models for Meta-analysis. Stats Med 1999; 18:6 :643-654. 41. Rosenthal R. Meta-analytic procedures for social research . 1991; 42. Schmid CH, Lau J, McIntosh MW, Cappelleri JC. An Empirical Study of the Effect of the Control rate as a Predictor of Treatment Efficacy in Meta-analysis of Clinical Trials. Stats Med 1998; 17:17 :1923-1943. 43. Smith TC, Spiegelhalter DJ, Thomas A. Bayesian Approaches to Random-effects Meta-analysis: A Comparative Study. Stats Med 1995; 14:24 :2685-2700. 44. Sullivan KM, Foster DA. Use of the Confidence Interval Function. Epidemiology 1990; 1:1 :39-42. 45. Thompson SG, Smith TC, Sharp SJ. Investigating Underlying Risk as a Source of Heterogeneity in Meta-analysis. Stats Med 1997; 16:23 :2741-2758. 46. Thompson SG, Pyke SDM, Hardy RJ. The design and analysis of paired cluster randomizede trials: an application of Meta-analysis Techniques. Stats Med 1997; 16:18 :2063-2080. 47. Thompson SG, Sharp SJ. Explaining Heterogeneity in Meta-analysis: A Comparison of Methods. Stats Med 1999; 18:20 :2693-2708. 48. Tritchler D. Modelling Study Quality in Meta-analysis. Stats Med 1999; 18:16 :2135-2146. 49. Van Houwelingen HC. Meta-analysis; methods, limitations. Byocybernetics and Biomedical Engineering 1995; 15 :53-61. 50. Walter SD. Variation in Baseline Risk as an Explanation of Heterogeneity in Meta-analysis. Stats Med 1997; 16:24 :2883-2900. 51. Walter SD, Jadad AR. Meta-analysis of Screening Data: A Survey of the Literature. Stats Med 1999; 18:24 :3409-3424. 52. Whitehead A, Whitehead J. A General Parametric Approach to the Meta-Analysis of Randomized Clinical Trials. Stats Med 1991; 10 :1665-1677. 53. Whitehead A, Whitehead J. A general parametric approach to the meta-analysis of randomized clinical trials. Stats Med 1991; 10 :1655-1677. 54. Whitehead A, Jones NMB. A Meta-analysis of Clinical Trials Involving Different Classifications of Response into Ordered Categories. Stats Med 1994; 13:23/24 :2503-2516. 55. Whitehead A. A Prospectively Planned Cumulative Meta-analysis Applied to a Series of Concurrent Clinical Trials. Stats Med 1997; 16:24 :2901-2914.