Skip to content
naulon
Get started

No retraction, correction or concern found when last checked on Oct 29, 2026.

Consistency and inconsistency in network meta‐analysis: concepts and models for multi‐arm studies

J. P. T. Higgins, D Jackson, J. K. Barrett and 3 more

Research Synthesis Methods · 2012

doi:10.1002/jrsm.1044

Abstract

Meta‐analyses that simultaneously compare multiple treatments (usually referred to as network meta‐analyses or mixed treatment comparisons) are becoming increasingly common. An important component of a network meta‐analysis is an assessment of the extent to which different sources of evidence are compatible, both substantively and statistically. A simple indirect comparison may be confounded if the studies involving one of the treatments of interest are fundamentally different from the studies involving the other treatment of interest. Here, we discuss methods for addressing inconsistency of evidence from comparative studies of different treatments. We define and review basic concepts of heterogeneity and inconsistency, and attempt to introduce a distinction between ‘loop inconsistency’ and ‘design inconsistency’. We then propose that the notion of design‐by‐treatment interaction provides a useful general framework for investigating inconsistency. In particular, using design‐by‐treatment interactions successfully addresses complications that arise from the presence of multi‐arm trials in an evidence network. We show how the inconsistency model proposed by Lu and Ades is a restricted version of our full design‐by‐treatment interaction model and that there may be several distinct Lu–Ades models for any particular data set. We introduce novel graphical methods for depicting networks of evidence, clearly depicting multi‐arm trials and illustrating where there is potential for inconsistency to arise. We apply various inconsistency models to data from trials of different comparisons among four smoking cessation interventions and show that models seeking to address loop inconsistency alone can run into problems. Copyright © 2012 John Wiley & Sons, Ltd.

Shown for reading. Its publisher allows quoting it, not republishing it.

References 21

  1. 1.

    Caldwell DM, Ades AE, Higgins JPT. Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ. 2005;331:897-900.

    Not in our library10.1136/bmj.331.7521.897

  2. 2.

    Chan AW, Altman DG. Epidemiology and reporting of randomised trials published in PubMed journals. The Lancet. 2005;365:1159-1162.

    Not in our library10.1016/s0140-6736(05)71879-1

  3. 3.

    Cooper NJ, Sutton AJ, Morris D, Ades AE, Welton NJ. Addressing between-study heterogeneity and inconsistency in mixed treatment comparisons: application to stroke prevention treatments in individuals with non-rheumatic atrial fibrillation. Statistics in Medicine. 2009;28:1861-1881.

    Not in our library10.1002/sim.3594

  4. 4.

    Dias S, Welton NJ, Caldwell DM, Ades AE. Checking consistency in mixed treatment comparison meta-analysis. Statistics in Medicine. 2010;29:932-944.

    Not in our library10.1002/sim.3767

  5. 5.

    Gartlehner G, Moore CG. Direct versus indirect comparisons: a summary of the evidence. International Journal of Technology Assessment in Health Care. 2008;24:170-177.

    Not in our library10.1017/s0266462308080240

  6. 6.

    Gleser LJ, Olkin I, Cooper H, Hedges LV. Stochastically dependent effect sizes. The Handbook of Research Synthesis. 1994;339-356.

    Not in our library

  7. 7.

    Hasselblad V. Meta-analysis of multitreatment studies. Medical Decision Making. 1998;18:37-43.

    Not in our library10.1177/0272989x9801800110

  8. 8.

    Higgins JPT. Commentary: heterogeneity in meta-analysis should be expected and appropriately quantified. International Journal of Epidemiology. 2008;37:1158-1160.

    Not in our library10.1093/ije/dyn204

  9. 9.

    Higgins JPT, Whitehead A. Borrowing strength from external trials in a meta-analysis. Statistics in Medicine. 1996;15:2733-2749.

    Not in our library10.1002/(sici)1097-0258(19961230)15:24<2733::aid-sim562>3.0.co;2-0

  10. 10.

    Ioannidis JPA. Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. Canadian Medical Association Journal. 2009;181:488-493.

    Not in our library10.1503/cmaj.081086

  11. 11.

    Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Statistics in Medicine. 2004;23:3105-3124.

    Not in our library10.1002/sim.1875

  12. 12.

    Lu G, Ades AE. Assessing evidence inconsistency in mixed treatment comparisons. Journal of the American Statistical Association. 2006;101:447-459.

    Not in our library

  13. 13.

    Lu G, Ades A. Modeling between-trial variance structure in mixed treatment comparisons. Biostatistics. 2009;10:792-805.

    Not in our library10.1093/biostatistics/kxp032

  14. 14.

    Lu G, Welton NJ, Higgins JPT, White IR, Ades AE. Linear inference for mixed treatment comparison meta-analysis: a two-stage approach. Research Synthesis Methods. 2011;2:43-60.

    Not in our library10.1002/jrsm.34

  15. 15.

    Lumley T. Network meta-analysis for indirect treatment comparisons. Statistics in Medicine. 2002;21:2313-2324.

    Not in our library10.1002/sim.1201

  16. 16.

    O'Regan C, Ghement I, Eyawo O, Guyatt GH, Mills EJ. Incorporating multiple interventions in meta-analysis: an evaluation of the mixed treatment comparison with the adjusted indirect comparison. Trials. 2009;10:86.

    Not in our library10.1186/1745-6215-10-86

  17. 17.

    Salanti G, Higgins JPT, Ades AE, Ioannidis JP. Evaluation of networks of randomized trials. Statistical Methods in Medical Research. 2008;17:279-301.

    Not in our library10.1177/0962280207080643

  18. 18.

    Salanti G, Marinho V, Higgins JPT. A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered. Journal of Clinical Epidemiology. 2009;62:857-864.

    Not in our library10.1016/j.jclinepi.2008.10.001

  19. 19.

    Song F, Xiong T, Parekh-Bhurke S, Loke YK, Sutton AJ, Eastwood AJ, Holland R, Chen YF, Glenny AM, Deeks JJ, Altman DG. Inconsistency between direct and indirect comparisons of competing interventions: meta-epidemiological study. BMJ. 2011;343:d4909.

    Not in our library10.1136/bmj.d4909

  20. 20.

    White IR. Multivariate random-effects meta-regression. The Stata Journal. 2011;11:255-270.

    Not in our library

  21. 21.
    Consistency and inconsistency in network meta‐analysis: model estimation using multivariate meta‐regression · 2012
    No notice

Cited by papers we hold 34

and 14 more

Reads

Read by people on this page

None yet.

Checked or read by AI through naulon

None yet.

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.

Are you an author?

Is this your paper?

Claim it to add your note and be paid when agents cite it. Free.