Sequential methods for random‐effects meta‐analysis

Abstract

Although meta‐analyses are typically viewed as retrospective activities, they are increasingly being applied prospectively to provide up‐to‐date evidence on specific research questions. When meta‐analyses are updated account should be taken of the possibility of false‐positive findings due to repeated significance tests. We discuss the use of sequential methods for meta‐analyses that incorporate random effects to allow for heterogeneity across studies. We propose a method that uses an approximate semi‐Bayes procedure to update evidence on the among‐study variance, starting with an informative prior distribution that might be based on findings from previous meta‐analyses. We compare our methods with other approaches, including the traditional method of cumulative meta‐analysis, in a simulation study and observe that it has Type I and Type II error rates close to the nominal level. We illustrate the method using an example in the treatment of bleeding peptic ulcers. Copyright © 2010 John Wiley & Sons, Ltd.

Shown for reading. Its publisher allows quoting it, not republishing it.

References 42

1. 1.

   Cochrane Database of Systematic Reviews. The Cochrane Library. 2010.

   Not in our library

2. 2.

   Higgins JPT, Green S, Scholten RJPM, Higgins JPT, Green S. Maintaining reviews: updates, amendments and feedback. Cochrane Handbook for Systematic Reviews of Interventions. 2008;31-49.

   Not in our library

3. 3.

   Simes RJ. Prospective meta-analysis of cholesterol-lowering studies: the prospective pravastatin pooling project (PPP) and the cholesterol treatment trialists (CTT) collaboration. American Journal of Cardiology. 1995;76:C122-C126.

   Not in our library10.1016/s0002-9149(99)80482-2

4. 4.

   Baigent C, Keech A, Kearney PM, Blackwell L, Buck G, Pollicino C, Kirby A, Sourjina T, Peto R, Collins R, Simes R. Efficacy and safety of cholesterol-lowering treatment: prospective meta-analysis of data from 90 056 participants in 14 randomised trials of statins. The Lancet. 2005;366:1267-1278.

   Not in our library10.1016/s0140-6736(05)67394-1

5. 5.

   Writer WD, Stienstra R, Eddleston JM, Gatt SP, Griffin R, Gutsche BB, Joyce TH, Hedlund C, Heeroma K, Selander D. Neonatal outcome and mode of delivery after epidural analgesia for labour with ropivacaine and bupivacaine: a prospective meta-analysis. British Journal of Anaesthesia. 1998;81:713-717.

   Not in our library10.1093/bja/81.5.713

6. 6.

   Schunkert H, Gotz A, Braund P, McGinnis R, Tregouet DA, Mangino M, Linsel-Nitschke P, Cambien F, Hengstenberg C, Stark K, Blankenberg S, Tiret L, Ducimetiere P, Keniry A, Ghori MJ, Schreiber S, El Mokhtari NE, Hall AS, Dixon RJ, Goodall AH, Liptau H, Pollard H, Schwarz DF, Hothorn LA, Wichmann HE, Konig IR, Fischer M, Meisinger C, Ouwehand W, Deloukas P, Thompson JR, Erdmann J, Ziegler A, Samani NJ. Repeated replication and a prospective meta-analysis of the association between chromosome 9p21.3 and coronary artery disease. Circulation. 2008;117:1675-1684.

   Not in our library10.1161/circulationaha.107.730614

7. 7.

   Sims AM, Timms AE, Bruges-Armas J, Burgos-Vargas R, Chou CT, Doan T, Dowling A, Fialho RN, Gergely P, Gladman DD, Inman R, Kauppi M, Kaarela K, Laiho K, Maksymowych W, Pointon JJ, Rahman P, Reveille JD, Tuomilehto J, Vargas-Alarcon G, Wordsworth BP, Xu H, Brown MA. Prospective meta-analysis of interleukin 1 gene complex polymorphisms confirms associations with ankylosing spondylitis. Annals of the Rheumatic Diseases. 2008;67:1305-1309.

   Not in our library10.1136/ard.2007.081364

8. 8.

   Berkey CS, Mosteller F, Lau J, Antman EM. Uncertainty of the time of first significance in random effects cumulative metaanalysis. Controlled Clinical Trials. 1996;17:357-371.

   Not in our library10.1016/s0197-2456(96)00014-1

9. 9.

   Whitehead A. A prospectively planned cumulative meta-analysis applied to a series of concurrent clinical trials. Statistics in Medicine. 1997;16:2901-2913.

   Not in our library10.1002/(sici)1097-0258(19971230)16:24<2901::aid-sim700>3.0.co;2-5

10. 10.

    Whitehead J. The Design and Analysis of Sequential Clinical Trials. 1997.

    Not in our library

11. 11.

    Pogue JM, Yusuf S. Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta-analysis. Controlled Clinical Trials. 1997;18:580-593.

    Not in our library10.1016/s0197-2456(97)00051-2

12. 12.

    Devereaux PJ, Beattie WS, Choi PT, Badner NH, Guyatt GH, Villar JC, Cina CS, Leslie K, Jacka MJ, Montori VM, Bhandari M, Avezum A, Cavalcanti AB, Giles JW, Schricker T, Yang H, Jakobsen CJ, Yusuf S. How strong is the evidence for the use of perioperative beta blockers in non-cardiac surgery? Systematic review and meta-analysis of randomised controlled trials. BMJ. 2005;331:313-321.

    Not in our library10.1136/bmj.38503.623646.8f

13. 13.

    Lan KKG, Hu M, Cappelleri JC. Applying the law of iterated logarithm to cumulative meta-analysis of a continuous endpoint. Statistica Sinica. 2003;13:1135-1145.

    Not in our library

14. 14.

    Hu M, Cappelleri JC, Lan KKG. Applying the law of iterated logarithm to control type I error in cumulative meta-analysis of binary outcomes. Clinical Trials. 2007;4:329-340.

    Not in our library10.1177/1740774507081219

15. 15.

    Brok J, Thorlund K, Gluud C, Wetterslev J. Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta-analyses. Journal of Clinical Epidemiology. 2008;61:763-769.

    Not in our library10.1016/j.jclinepi.2007.10.007

16. 16.

    Wetterslev J, Thorlund K, Brok J, Gluud C. Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis. Journal of Clinical Epidemiology. 2008;61:64-75.

    Not in our library10.1016/j.jclinepi.2007.03.013

17. 17.

    PEST 4. 2000.

    Not in our library

18. 18.

    Whitehead A, Whitehead J. A general parametric approach to the meta-analysis of randomised clinical trials. Statistics in Medicine. 1991;10:1665-1677.

    Not in our library10.1002/sim.4780101105

19. 19.

    Shuster JJ. Empirical vs natural weighting in random effects meta-analysis. Statistics in Medicine. 2010;29:1259-1265.

    Not in our library10.1002/sim.3607

20. 20.

    Laird N, Fitzmaurice G, Ding X. Comments on ‘Empirical vs natural weighting in random effects meta-analysis’. Statistics in Medicine. 2010;29:1266-1267.

    Not in our library10.1002/sim.3657

21. 21.

    Waksman JA. Comments on ‘Empirical vs natural weighting in random effects meta-analysis’. Statistics in Medicine. 2010;29:1268-1269.

    Not in our library10.1002/sim.3692

22. 22.

    Thompson SG, Higgins JPT. Comments on ‘Empirical vs natural weighting in random effects meta-analysis’. Statistics in Medicine. 2010;29:1270-1271.

    Not in our library10.1002/sim.3718

23. 23.

    DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials. 1986;7:177-188.

    Not in our library10.1016/0197-2456(86)90046-2

24. 24.
    A Re-Evaluation of Random-Effects Meta-Analysis · 2008

    No notice
25. 25.

    Bollen CW, Uiterwaal CS, van Vught AJ, van dT I. Sequential meta-analysis of past clinical trials to determine the use of a new trial. Epidemiology. 2006;17:644-649.

    Not in our library10.1097/01.ede.0000239658.19288.22

26. 26.

    O'Brien PC, Fleming TR. A multiple testing procedure for clinical trials. Biometrics. 1979;35:549-556.

    Not in our library

27. 27.

    Jennison C, Turnbull BW. Interim analyses: the repeated confidence interval approach. Journal of the Royal Statistical Society, Series B. 1989;51:305-361.

    Not in our library

28. 28.

    Biggerstaff BJ, Tweedie RL. Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis. Statistics in Medicine. 1997;16:753-768.

    Not in our library10.1002/(sici)1097-0258(19970415)16:7<753::aid-sim494>3.0.co;2-g

29. 29.

    Hardy RJ, Thompson SG. A likelihood approach to meta-analysis with random effects. Statistics in Medicine. 1996;15:619-629.

    Not in our library10.1002/(sici)1097-0258(19960330)15:6<619::aid-sim188>3.0.co;2-a

30. 30.

    Higgins JPT, Whitehead A. Borrowing strength from external trials in a meta-analysis. Statistics in Medicine. 1996;15:2733-2749.

    Not in our library10.1002/(sici)1097-0258(19961230)15:24<2733::aid-sim562>3.0.co;2-0

31. 31.

    Cohen J. Statistical Power Analysis for the Behavioral Sciences. 1988.

    Not in our library

32. 32.

    Sacks HS, Chalmers TC, Blum AL, Berrier J, Pagano D. Endoscopic hemostasis: an effective therapy for bleeding peptic ulcers. Journal of the American Medical Association. 1990;264:494-499.

    Not in our library10.1001/jama.264.4.494

33. 33.

    Bradburn MJ, Deeks JJ, Berlin JA, Localio AR. Much ado about nothing: a comparison of the performance of meta-analytical methods with rare events. Statistics in Medicine. 2006;26:53-77.

    Not in our library10.1002/sim.2528

34. 34.

    Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315:629-634.

    Not in our library10.1136/bmj.315.7109.629

35. 35.

    Harbord RM, Egger M, Sterne JA. A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints. Statistics in Medicine. 2006;25:3443-3457.

    Not in our library10.1002/sim.2380

36. 36.

    Deeks JJ, Altman DG, Egger M, Davey Smith G, Altman DG. Effect measures for meta-analysis of trials with binary outcomes. Systematic Reviews in Health Care: Meta-analysis in Context. 2001;313-335.

    Not in our library

37. 37.

    Chalmers TC, Lau J. Meta-analytic stimulus for changes in clinical trials. Statistical Methods in Medical Research. 1993;2:161-172.

    Not in our library10.1177/096228029300200204

38. 38.

    Sutton AJ, Cooper NJ, Jones DR, Lambert PC, Thompson JR, Abrams KR. Evidence-based sample size calculations based upon updated meta-analysis. Statistics in Medicine. 2006;26:2479-2500.

    Not in our library10.1002/sim.2704

39. 39.

    Stallard N, Facey KM. Comparison of the spending function methods and the christmas tree correction for group sequential trials. Journal of Biopharmaceutical Statistics. 1996;6:361-373.

    Not in our library10.1080/10543409608835149

40. 40.

    Berry DA. Interim analyses in clinical-trials: classical vs bayesian approaches. Statistics in Medicine. 1985;4:521-526.

    Not in our library10.1002/sim.4780040412

Cited by papers we hold 9

• Power analysis for random‐effects meta‐analysis · 2017
  No notice
• Trial Sequential Analysis in systematic reviews with meta-analysis · 2017
  No notice
• Regional citrate versus heparin anticoagulation for continuous renal replacement therapy in critically ill patients: a meta-analysis with trial sequential analysis of randomized controlled trials · 2016
  No notice
• Living Systematic Reviews: An Emerging Opportunity to Narrow the Evidence-Practice Gap · 2014
  No notice
• Lower Pill Burden and Once-Daily Antiretroviral Treatment Regimens for HIV Infection: A Meta-Analysis of Randomized Controlled Trials · 2014
  No notice
• The thresholds for statistical and clinical significance – a five-step procedure for evaluation of intervention effects in randomised clinical trials · 2014
  No notice
• Thresholds for statistical and clinical significance in systematic reviews with meta-analytic methods · 2014
  No notice
• Sample size and power considerations in network meta-analysis · 2012
  No notice
• The Number of Patients and Events Required to Limit the Risk of Overestimation of Intervention Effects in Meta-Analysis—A Simulation Study · 2011
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.