Multivariate meta‐analysis: Potential and promise

Abstract

The multivariate random effects model is a generalization of the standard univariate model. Multivariate meta‐analysis is becoming more commonly used and the techniques and related computer software, although continually under development, are now in place. In order to raise awareness of the multivariate methods, and discuss their advantages and disadvantages, we organized a one day ‘Multivariate meta‐analysis’ event at the Royal Statistical Society. In addition to disseminating the most recent developments, we also received an abundance of comments, concerns, insights, critiques and encouragement. This article provides a balanced account of the day's discourse. By giving others the opportunity to respond to our assessment, we hope to ensure that the various view points and opinions are aired before multivariate meta‐analysis simply becomes another widely used de facto method without any proper consideration of it by the medical statistics community. We describe the areas of application that multivariate meta‐analysis has found, the methods available, the difficulties typically encountered and the arguments for and against the multivariate methods, using four representative but contrasting examples. We conclude that the multivariate methods can be useful, and in particular can provide estimates with better statistical properties, but also that these benefits come at the price of making more assumptions which do not result in better inference in every case. Although there is evidence that multivariate meta‐analysis has considerable potential, it must be even more carefully applied than its univariate counterpart in practice. Copyright © 2011 John Wiley & Sons, Ltd.

Shown for reading. Its publisher allows quoting it, not republishing it.

References 75

1. 1.

   Egger M, Smith GD. Misleading meta-analysis. British Medical Journal. 1995;310:752-754.

   Not in our library10.1136/bmj.310.6982.752

2. 2.

   Egger M, Smith GD. Meta-analysis: potentials and promise. British Medical Journal. 1997;315:1371-1375.

   Not in our library10.1136/bmj.315.7119.1371

3. 3.

   Eysenck HJ. Meta-analysis and its problems. British Medical Journal. 1994;309:789-792.

   Not in our library10.1136/bmj.309.6957.789

4. 4.

   Normand ST. Meta-analysis: formulating, evaluating, combining and reporting. Statistics in Medicine. 1999;18:321-359.

   Not in our library10.1002/(sici)1097-0258(19990215)18:3<321::aid-sim28>3.0.co;2-p

5. 5.

   Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. Journal of Clinical Epidemiology. 2005;58:982-990.

   Not in our library10.1016/j.jclinepi.2005.02.022

6. 6.

   Higgins JPT, Whitehead A, Turner RM, Omar RZ, Thompson SG. Meta-analysis of continuous outcome data from individual patients. Statistics in Medicine. 2001;20:2219-2241.

   Not in our library10.1002/sim.918

7. 7.

   Turner RM, Omar RZ, Yang M, Goldstein H, Thompson SG. A multilevel model framework for meta-analysis of clinical trials with binary outcomes. Statistics in Medicine. 2000;19:3417-3432.

   Not in our library10.1002/1097-0258(20001230)19:24<3417::aid-sim614>3.0.co;2-l

8. 8.

   Whitehead A, Omar RZ, Higgins JPT, Savaluny E, Turner RM, Thompson SG. Meta-analysis of ordinal outcomes using individual patient data. Statistics in Medicine. 2001;20:2241-2243.

   Not in our library10.1002/sim.919

9. 9.

   Tudur Smith C, Williamson PR, Marson AG. Investigating heterogeneity in an individual patient data meta-analysis of time to event outcomes. Statistics in Medicine. 2005;24:1307-1319.

   Not in our library10.1002/sim.2050

10. 10.

    Systematically missing confounders in individual participant data meta-analysis of observational studies. Statistics in Medicine. 2009;28:1218-1237.

    Not in our library10.1002/sim.3540

11. 11.

    van Houwelingen HC, Arends LR, Stijnen T. Advanced methods in meta-analysis: multivariate approach and meta-regression. Statistics in Medicine. 2002;21:589-624.

    Not in our library10.1002/sim.1040

12. 12.

    Ishak KJ, Platt RW, Joseph L, Hanley JA. Impact of approximating or ignoring within-study covariances in multivariate meta-analyses. Statistics in Medicine. 2008;27:670-686.

    Not in our library10.1002/sim.2913

13. 13.

    Riley R. Multivariate meta-analysis: the effect of ignoring within-study correlation. Journal of the Royal Statistical Society, Series A. 2009;172:789-811.

    Not in our library

14. 14.

    White IR. Multivariate meta-analysis. The Stata Journal. 2009;9:40-56.

    Not in our library

15. 15.

    Gleser LJ, Olkin I, Cooper H, Hedges LV. Stochastically dependent effect sizes. The Handbook of Research Synthesis. 1994.

    Not in our library

16. 16.

    Raudenbush SW, Becker BJ, Kalaian H. Modeling multivariate effect sizes. Psychological Bulletin. 1988;1039:111-120.

    Not in our library

17. 17.

    Kalaian HA, Raudenbush SW. A multivariate mixed linear model for meta-analysis. Psychological Methods. 1996;1:227-235.

    Not in our library

18. 18.

    Hedges LV, Olkin I. Statistical Methods for Meta-analysis. 1994.

    Not in our library

19. 19.

    Berkey CS, Antczak-Bouckoms A, Hoaglin DC, Mosteller F, Pihlstrom BL. Multiple outcomes meta-analysis of treatments for periodontal disease. Journal of Dental Research. 1995;74:1030-1039.

    Not in our library10.1177/00220345950740040201

20. 20.

    Berkey CS, Hoaglin DC, Antczak-Bouckoms A, Mosteller F, Colditz GA. Meta-analysis of multiple outcomes by regression with random effects. Statistics in Medicine. 1998;17:2537-2550.

    Not in our library10.1002/(sici)1097-0258(19981130)17:22<2537::aid-sim953>3.0.co;2-c

21. 21.

    Sohn SY. Multivariate meta-analysis with potentially correlated marketing study results. Naval Research Logistics. 2000;47:500-510.

    Not in our library

22. 22.

    Daniels MJ, Hughes MD. Meta-analysis for the evaluation of potential surrogate markers. Statistics in Medicine. 1997;16:1965-1982.

    Not in our library10.1002/(sici)1097-0258(19970915)16:17<1965::aid-sim630>3.0.co;2-m

23. 23.

    Gail MH, Pfeiffer R, van Houwelingen HC, Carroll RJ. On meta-analytic assessment of surrogate outcomes. Biostatistics. 2000;1:231-246.

    Not in our library10.1093/biostatistics/1.3.231

24. 24.

    Thompson JR, Minelli C, Abrams KR, Tobin MD, Riley RD. Meta-analysis of genetic studies using Mendelian randomization—a multivariate approach. Statistics in Medicine. 2005;24:2241-2254.

    Not in our library10.1002/sim.2100

25. 25.

    Riley RD, Abrams KR, Lambert PC, Sutton AJ, Thompson JR. An evaluation of bivariate random effects meta-analysis for the joint synthesis of two correlated outcomes. Statistics in Medicine. 2007;26:78-97.

    Not in our library10.1002/sim.2524

26. 26.

    Hamza TH, Arends LR, van Houwelingen HC, Stijnen T. Multivariate random effects meta-analysis of diagnostic tests with multiple thresholds. BMC Medical Research Methodology. 2009;9:73.

    Not in our library10.1186/1471-2288-9-73

27. 27.

    Jones AP, Riley RD, Williamson PR, Whitehead A. Meta-analysis of longitudinal data. Clincal Trials. 2009;6:16-27.

    Not in our library10.1177/1740774508100984

28. 28.

    Chu H, Cole SR. Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. Journal of Clinical Epidemiology. 2006;59:1331-1333.

    Not in our library10.1016/j.jclinepi.2006.06.011

29. 29.

    Hamza TH, van Houwelingen HC, Stijnen T. The binomial distribution of meta-analysis was preferred to model within-study variability. Journal of Clinical Epidemiology. 2008;61:41-51.

    Not in our library10.1016/j.jclinepi.2007.03.016

30. 30.

    Harbord RM, Whiting P. Metandi: meta-analysis of diagnostic accuracy using hierarchical logistic regression. The Stata Journal. 2009;9:211-229.

    Not in our library

31. 31.

    Takwoingi Y. 2008.

    Not in our library

32. 32.

    Harbord RM, Deeks JJ, Egger M, Whiting P, Sterne JAC. A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics. 2007;8:239-251.

    Not in our library10.1093/biostatistics/kxl004

33. 33.

    Rutter CM, Gatsonis CA. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Statistics in Medicine. 2001;20:2865-2884.

    Not in our library10.1002/sim.942

34. 34.

    Putter H, Fiocco M, Stijnen T. Meta-analysis of diagnostic test accuracy studies with multiple thresholds using survival methods. Biometrical Journal. 2010;52:95-110.

    Not in our library10.1002/bimj.200900073

35. 35.

    Chu H, Nie L, Cole SR, Poole C. Meta-analysis of diagnostic accuracy studies accounting for disease prevalence: alternative parameterizations and model selection. Statistics in Medicine. 2009;28:2384-2399.

    Not in our library10.1002/sim.3627

36. 36.

    Paul M, Riebler A, Bachmann LM, Rue H, Held L. Bayesian bivariate meta-analysis of diagnostic test studies using integrated nested Laplace approximations. Statistics in Medicine. 2010;29:1325-1339.

    Not in our library10.1002/sim.3858

37. 37.

    Kertai MD, Boersma E, Bax JJ, Heijenbrok-Kal MH, Hunink MGM, L'talien GJ, Roelandt JRTC, van Urk H, Poldermans D. A meta-analysis comparing the prognostic accuracy of six diagnostic tests for predicting perioperative cardiac risk in patients undergoing major vascular surgery. Heart. 2003;89:1327-1334.

    Not in our library10.1136/heart.89.11.1327

38. 38.

    Berkey CS, Anderson JJ, Hoaglin DC. Multiple-outcome meta-analysis of clinical trials. Statistics in Medicine. 1996;15:537-557.

    Not in our library10.1002/(sici)1097-0258(19960315)15:5<537::aid-sim176>3.0.co;2-s

39. 39.

    Arends LR, Hunink MG, Stijnen T. Meta-analysis of summary survival curve data. Statistics in Medicine. 2008;27:4381-4396.

    Not in our library10.1002/sim.3311

40. 40.

    van Houwelingen HC, Zwinderman KH, Stijnen T. A bivariate approach to meta-analysis. Statistics in Medicine. 1993;12:2273-2284.

    Not in our library10.1002/sim.4780122405

Cited by papers we hold 18

• Caffeine, Coffee, Tea and Risk of Rheumatoid Arthritis: Systematic Review and Dose-Response Meta-analysis of Prospective Cohort Studies · 2022
  Not checked
• Mindfulness-based programmes for mental health promotion in adults in nonclinical settings: A systematic review and meta-analysis of randomised controlled trials · 2021
  No notice
• A multi-cohort consortium for GEnder-Sensitive Analyses of mental health trajectories and implications for prevention (GESA) in the general population in Germany · 2020
  Not checked
• Individual participant data meta‐analysis to examine interactions between treatment effect and participant‐level covariates: Statistical recommendations for conduct and planning · 2020
  No notice
• What exercise prescription is optimal to improve body composition and cardiorespiratory fitness in adults living with obesity? A network meta‐analysis · 2020
  No notice
• A Guide to Conducting a Meta-Analysis with Non-Independent Effect Sizes · 2019
  No notice
• When should meta‐analysis avoid making hidden normality assumptions? · 2018
  No notice
• Multivariate and network meta-analysis of multiple outcomes and multiple treatments: rationale, concepts, and examples · 2017
  No notice
• Meta-evaluation of meta-analysis: ten appraisal questions for biologists · 2017
  No notice
• Meta-analysis using individual participant data: one-stage and two-stage approaches, and why they may differ · 2016
  No notice
• Accuracy of non-invasive prenatal testing using cell-free DNA for detection of Down, Edwards and Patau syndromes: a systematic review and meta-analysis · 2016
  No notice
• A graphical tool for locating inconsistency in network meta-analyses · 2013
  No notice
• Individual Participant Data Meta-Analysis for a Binary Outcome: One-Stage or Two-Stage? · 2013
  No notice
• Graphical Tools for Network Meta-Analysis in STATA · 2013
  No notice
• Reducing and meta-analysing estimates from distributed lag non-linear models · 2013
  No notice
• Quantifying the impact of between‐study heterogeneity in multivariate meta‐analyses · 2012
  No notice
• Multivariate meta‐analysis for non‐linear and other multi‐parameter associations · 2012
  No notice
• Consistency and inconsistency in network meta‐analysis: model estimation using multivariate meta‐regression · 2012
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.