Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests

Abstract

The confirmatory analysis of pre‐specified multiple hypotheses has become common in pivotal clinical trials. In the recent past multiple test procedures have been developed that reflect the relative importance of different study objectives, such as fixed sequence, fallback, and gatekeeping procedures. In addition, graphical approaches have been proposed that facilitate the visualization and communication of Bonferroni‐based closed test procedures for common multiple test problems, such as comparing several treatments with a control, assessing the benefit of a new drug for more than one endpoint, combined non‐inferiority and superiority testing, or testing a treatment at different dose levels in an overall and a subpopulation. In this paper, we focus on extended graphical approaches by dissociating the underlying weighting strategy from the employed test procedure. This allows one to first derive suitable weighting strategies that reflect the given study objectives and subsequently apply appropriate test procedures, such as weighted Bonferroni tests, weighted parametric tests accounting for the correlation between the test statistics, or weighted Simes tests. We illustrate the extended graphical approaches with several examples. In addition, we describe briefly the gMCP package in R , which implements some of the methods described in this paper.

Shown for reading. Its publisher allows quoting it, not republishing it.

References 49

1. 1.

   Alosh M, Huque MF. A flexible strategy for testing subgroups and overall population. Statistics in Medicine. 2009;28:3-23.

   Not in our library10.1002/sim.3461

2. 2.

   Alosh M, Huque MF. A consistency-adjusted alpha-adaptive strategy for sequential testing. Statistics in Medicine. 2010;29:1559-1571.

   Not in our library10.1002/sim.3896

3. 3.

   Bauer P. Multiple testing in clinical trials. Statistics in Medicine. 1991;10:871-890.

   Not in our library10.1002/sim.4780100609

4. 4.

   Bauer P, Röhmel J, Maurer W, Hothorn L. Testing strategies in multi-dose experiments including active control. Statistics in Medicine. 1998;17:2133-2146.

   Not in our library10.1002/(sici)1097-0258(19980930)17:18<2133::aid-sim901>3.0.co;2-2

5. 5.

   Benjamini Y, Heller R. False discovery rates for spatial signals. Journal of the American Statistical Association. 2007;102:1272-1281.

   Not in our library

6. 6.

   Benjamini Y, Hochberg Y. Multiple hypothesis testing with weights. Scandinavian Journal of Statistics. 1997;24:407-418.

   Not in our library

7. 7.

   Benjamini Y, Yekutieli D. The control of the false discovery rate in multiple testing under dependency. Annals of Statistics. 2001;29:1165-1188.

   Not in our library

8. 8.

   Brannath W, Bretz F. Shortcuts for locally consonant closed test procedures. Journal of the American Statistical Association. 2010;105:660-669.

   Not in our library

9. 9.

   Bretz F, Maurer W, Brannath W, Posch M. A graphical approach to sequentially rejective multiple test procedures. Statistics in Medicine. 2009;28:586-604.

   Not in our library10.1002/sim.3495

10. 10.

    Bretz F, Maurer W, Hommel G. Test and power considerations for multiple endpoint analyses using sequentially rejective graphical procedures. Statistics in Medicine. 2011;30:1489-1501.

    Not in our library10.1002/sim.3988

11. 11.

    Burman CF, Sonesson C, Guilbaud O. A recycling framework for the construction of Bonferroni-based multiple tests. Statistics in Medicine. 2009;28:739-761.

    Not in our library10.1002/sim.3513

12. 12.

    Chen X, Luo X, Capizzi T. The application of enhanced parallel gatekeeping strategies. Statistics in Medicine. 2005;24:1385-1397.

    Not in our library10.1002/sim.2005

13. 13.

    Dmitrienko A, Offen W, Wang O, Xiao D. Gatekeeping procedures in dose-response clinical trials based on the Dunnett test. Pharmaceutical Statistics. 2006;5:19-28.

    Not in our library10.1002/pst.190

14. 14.

    Dmitrienko A, Offen WW, Westfall PH. Gatekeeping strategies for clinical trials that do not require all primary effects to be significant. Statistics in Medicine. 2003;22:2387-2400.

    Not in our library10.1002/sim.1526

15. 15.

    Dmitrienko A, Tamhane AC, Dmitrienko A, Tamhane AC, Bretz F. Gatekeeping procedures in clinical trials. Multiple Testing Problems in Pharmaceutical Statistics,. 2009.

    Not in our library

16. 16.

    Dmitrienko A, Tamhane A, Wiens B. General multi-stage gatekeeping procedures. Biometrical Journal. 2008;50:667-677.

    Not in our library10.1002/bimj.200710464

17. 17.

    Genz A, Bretz F. Computation of Multivariate Normal and t Probabilities. 2009.

    Not in our library

18. 18.

    Guilbaud O. Simultaneous confidence regions corresponding to Holm's stepdown procedure and other closed-testing procedures. Biometrical Journal. 2008;50:678-692.

    Not in our library10.1002/bimj.200710449

19. 19.

    Guilbaud O. Alternative confidence regions for Bonferroni-based closed-testing procedures that are not alpha-exhaustive. Biometrical Journal. 2009;51:721-735.

    Not in our library10.1002/bimj.200800113

20. 20.

    Guilbaud O, Karlsson P. Confidence regions for Bonferroni-based closed tests extended to more general closed tests. Journal of Biopharmaceutical Statistics. 2011;21:682-707.

    Not in our library10.1080/10543406.2011.551331

21. 21.

    Hochberg Y, Tamhane AC. Multiple Comparison Procedures. 1987.

    Not in our library

22. 22.

    Holm S. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics. 1979;6:65-70.

    Not in our library

23. 23.

    Hommel G. A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika. 1988;75:383-386.

    Not in our library

24. 24.

    Hommel G, Bretz F, Maurer W. Powerful short-cuts for multiple testing procedures with special reference to gatekeeping strategies. Statistics in Medicine. 2007;26:4063-4073.

    Not in our library10.1002/sim.2873

25. 25.

    Hung HMJ, Wang SJ. Some controversial multiple testing problems in regulatory applications. Journal of Biopharmaceutical Statistics. 2009;19:1-11.

    Not in our library10.1080/10543400802541693

26. 26.

    Hung HMJ, Wang SJ. Challenges to multiple testing in clinical trials. Biometrical Journal. 2010;52:747-756.

    Not in our library10.1002/bimj.200900206

27. 27.

    Huque MF, Alosh M. A flexible fixed-sequence testing method for hierarchically ordered correlated multiple endpoints in clinical trials. Journal of Statistical Planning and Inference. 2008;138:321-335.

    Not in our library

28. 28.

    Huque MF, Alosh M, Bhore R. Addressing multiplicity issues of a composite endpoint and its components in clinical trials. Journal of Biopharmaceutical Statistics. 2011;21:610-634.

    Not in our library10.1080/10543406.2011.551327

29. 29.

    Ihaka R, Gentleman R. R: A language for data analysis and graphics. Journal of Computational and Graphical Statistics. 1996;5:299-314.

    Not in our library

30. 30.

    Kling Y. 2005.

    Not in our library

31. 31.

    Lawrence J. Testing non-inferiority and superiority for two endpoints for several treatments with a control. Pharmaceutical Statistics. 2011.

    Not in our library10.1002/pst.468

32. 32.

    Li J, Mehrotra D. An efficient method for accommodating potentially underpowered primary endpoints. Statistics in Medicine. 2008;27:5377-5391.

    Not in our library10.1002/sim.3369

33. 33.

    Liu Y, Hsu J. Testing for efficacy in primary and secondary endpoints by partitioning decision paths. Journal of the American Statistical Association. 2009;104:1661-1670.

    Not in our library

34. 34.

    Marcus R, Peritz E, Gabriel KR. On closed testing procedure with special reference to ordered analysis of variance. Biometrika. 1976;63:655-660.

    Not in our library

35. 35.

    Maurer W, Glimm E, Bretz F. Multiple and repeated testing of primary, co-primary and secondary hypotheses. Statistics in Biopharmaceutical Research. 2011;3:336-352.

    Not in our library

36. 36.

    Maurer W, Hothorn L, Lehmacher W, Vollmar J. Multiple comparisons in drug clinical trials and preclinical assays: a-priori ordered hypotheses. Biometrie in der Chemisch-Pharmazeutischen Industrie. 1995;3-18.

    Not in our library

37. 37.

    Millen BA, Dmitrienko A. Chain procedures: A class of flexible closed testing procedures with clinical trial applications. Statistics in Biopharmaceutical Research. 2011;3:14-30.

    Not in our library

38. 38.

    O'Neill RT. Secondary endpoints cannot be validly analyzed if the primary endpoint does not demonstrate clear statistical significance. Controlled Clinical Trials. 1997;18:550-556.

    Not in our library10.1016/s0197-2456(97)00075-5

39. 39.

    R: A Language and Environment for Statistical Computing. 2011.

    Not in our library

40. 40.

    Rohmeyer K, Klinglmueller F. 2011.

    Not in our library

Cited by papers we hold 4

• MultipleTesting.com: A tool for life science researchers for multiple hypothesis testing correction · 2021
  Corrected
• Effects of once-weekly semaglutide vs once-daily canagliflozin on body composition in type 2 diabetes: a substudy of the SUSTAIN 8 randomised controlled clinical trial · 2020
  No notice
• Methods to adjust for multiple comparisons in the analysis and sample size calculation of randomised controlled trials with multiple primary outcomes · 2019
  Corrected
• Different Adjuvants Induce Common Innate Pathways That Are Associated with Enhanced Adaptive Responses against a Model Antigen in Humans · 2017
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.