Bias Modelling in Evidence Synthesis

Abstract

Summary Policy decisions often require synthesis of evidence from multiple sources, and the source studies typically vary in rigour and in relevance to the target question. We present simple methods of allowing for differences in rigour (or lack of internal bias) and relevance (or lack of external bias) in evidence synthesis. The methods are developed in the context of reanalysing a UK National Institute for Clinical Excellence technology appraisal in antenatal care, which includes eight comparative studies. Many were historically controlled, only one was a randomized trial and doses, populations and outcomes varied between studies and differed from the target UK setting. Using elicited opinion, we construct prior distributions to represent the biases in each study and perform a bias-adjusted meta-analysis. Adjustment had the effect of shifting the combined estimate away from the null by approximately 10%, and the variance of the combined estimate was almost tripled. Our generic bias modelling approach allows decisions to be based on all available evidence, with less rigorous or less relevant studies downweighted by using computationally simple methods.

Shown for reading. Its publisher allows quoting it, not republishing it.

References 46

1. 1.

   Ades AE, Sutton AJ. Multiparameter evidence synthesis in epidemiology and medical decision-making: current approaches. J. R. Statist. Soc. A. 2006;169:5-35.

   Not in our library

2. 2.

   Armitage P, Berry G. Statistical Methods in Medical Research. 2001.

   Not in our library

3. 3.

   Balk EM, Bonis P A L, Moskowitz H, Schmid CH, Ioannidis J P A, Wang C, Lau J. Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. J. Am. Med. Ass. 2002;287:2973-2982.

   Not in our library10.1001/jama.287.22.2973

4. 4.

   Bowman JM, Chown B, Lewis M, Pollock JM. Rh isoimmunization during pregnancy: antenatal prophylaxis. Can. Med. Ass. J. 1978;118:623-627.

   Not in our library

5. 5.

   Bowman JM, Pollock JM. Antenatal prophylaxis of Rh isoimmunization: 28-weeks’-gestation service program. Can. Med. Ass. J. 1978;118:627-630.

   Not in our library

6. 6.

   Bowman JM, Pollock JM. Failures of intravenous Rh immune globulin prophylaxis: an analysis of the reasons for such failures. Transfusn Med. Rev. 1987;1:101-112.

   Not in our library10.1016/s0887-7963(87)70010-8

7. 7.

   Chilcott J, Lloyd Jones M, Wight J, Forman K, Wray J, Beverley C, Tappenden P. A review of the clinical effectiveness and cost-effectiveness of routine anti-D prophylaxis for pregnant women who are rhesus-negative. Hlth Technol. Assmnt. 2003;7.

   Not in our library10.3310/hta7040

8. 8.

   Contreras M. The prevention of Rh haemolytic disease of the fetus and newborn—general background. Br. J. Obstetr. Gyn., suppl. 1998;18:7-10.

   Not in our library10.1111/j.1471-0528.1998.tb10285.x

9. 9.

   de Craen A J M, van Vliet H A A M, Helmerhorst FM. An analysis of systematic reviews indicated low incorporation of results from clinical trial quality assessment. J. Clin. Epidem. 2005;58:311-313.

   Not in our library10.1016/j.jclinepi.2004.07.002

10. 10.

    Deeks JJ, Dinnes J, D’Amico R, Sowden AJ, Sakarovitch C, Song F. Evaluating non-randomised intervention studies. Hlth Technol. Assmnt. 2003;7.

    Not in our library10.3310/hta7270

11. 11.

    DerSimonian R, Kacker R. Random-effects model for meta-analysis of clinical trials: an update. Contemp. Clin. Trials. 2007;28:105-114.

    Not in our library10.1016/j.cct.2006.04.004

12. 12.

    DerSimonian R, Laird N. Meta-analysis in clinical trials. Contr. Clin. Trials. 1986;7:177-188.

    Not in our library10.1016/0197-2456(86)90046-2

13. 13.

    Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care intervention. J. Epidem. Commty Hlth. 1998;52:377-384.

    Not in our library10.1136/jech.52.6.377

14. 14.

    Eddy DM, Hasselblad V, Schachter R. Meta-analysis by the Confidence Profile Method: the Statistical Synthesis of Evidence. 1992.

    Not in our library

15. 15.

    Evans M, Hastings N, Peacock B. Statistical Distributions. 1993.

    Not in our library

16. 16.

    Gluud LL. Bias in clinical intervention research. Am. J. Epidem. 2006;163:493-501.

    Not in our library10.1093/aje/kwj069

17. 17.

    Greenland S. The impact of prior distributions for uncontrolled confounding and response bias: a case study of the relation of wire codes and magnetic fields to childhood leukemia. J. Am. Statist. Ass. 2003;98:47-54.

    Not in our library

18. 18.

    Greenland S. Multiple-bias modelling for analysis of observational data (with discussion). J. R. Statist. Soc. A. 2005;168:267-306.

    Not in our library

19. 19.

    Greenland S, O’Rourke K. On the bias produced by quality scores in meta-analysis, and a hierarchical view of proposed solutions. Biostatistics. 2001;2:463-471.

    Not in our library10.1093/biostatistics/2.4.463

20. 20.

    Hermann M, Kjellman H, Ljunggren C. Antenatal prophylaxis of Rh immunization with 250 μg anti-D immunoglobulin. Acta Obstetr. Gyn. Scand. 1984;124(suppl):1-15.

    Not in our library10.3109/00016348409157011

21. 21.

    The Cochrane handbook for systematic reviews of interventions 4.2.5.

    Not in our library

22. 22.

    Higgins J P T, Thompson SG. Quantifying heterogeneity in a meta-analysis. Statist. Med. 2002;21:1539-1558.

    Not in our library10.1002/sim.1186

23. 23.

    Huchet J, Dallemagne S, Huchet C, Brossard Y, Larsen M, Parnet-Mathieu F. Application anté-partum du traitement préventif d'immunisation Rhésus D chez les femmes Rhésus négatif. J. Gyn. Obstetr. Biol. Reprodn. 1987;16:101-111.

    Not in our library

24. 24.

    Juni P, Altman DG, Egger M. Systematic review in health care: assessing the quality of controlled clinical trials. Br. Med. J. 2001;323:42-46.

    Not in our library10.1136/bmj.323.7303.42

25. 25.

    Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann. Intern. Med. 2001;135:982-989.

    Not in our library10.7326/0003-4819-135-11-200112040-00010

26. 26.

    Lee D, Rawlinson VI. Multicentre trial of antepartum low-dose anti-D immunoglobulin. Transfusn Med. 1995;5:15-19.

    Not in our library10.1111/j.1365-3148.1995.tb00180.x

27. 27.

    MacKenzie IZ, Bowell P, Gregory H, Pratt G, Guest C, Entwistle CC. Routine antenatal Rhesus D immunoglobulin prophylaxis: the results of a prospective 10 year study. Br. J. Obstetr. Gyn. 1999;106:492-497.

    Not in our library10.1111/j.1471-0528.1999.tb08304.x

28. 28.

    Mayne S, Parker JH, Harden TA, Dodds SD, Beale JA. Rate of RhD sensitisation before and after implementation of a community based antenatal prophylaxis programme. Br. Med. J. 1997;315:1588.

    Not in our library10.1136/bmj.315.7122.1588

29. 29.

    Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, Tugwell P, Klassen TP. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses. Lancet. 1998;352:609-613.

    Not in our library10.1016/s0140-6736(98)01085-x

30. 30.

    Moja LP, Telaro E, D’Amico R, Moschetti I, Coe L, Liberati A. Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study. Br. Med. J. 2005;330:1053-1055.

    Not in our library10.1136/bmj.38414.515938.8f

31. 31.

    Oakley JE, O’Hagan A. Uncertainty in prior elicitations: a nonparametric approach. Biometrika. 2007;94:427-441.

    Not in our library

32. 32.

    O’Hagan A, Buck CE, Daneshkhah A, Eiser JR, Garthwaite PH, Jenkinson DJ, Oakley JE, Rakow T. Uncertain Judgements: Eliciting Experts’ Probabilities. 2006.

    Not in our library

33. 33.

    Parsons ML, Van den Hof MC, Armson BA, Coffey J, Liston RM, Peddle LJ, Stinson DA. A comparison of the rate of Rh(D) alloimmunisation between Nova Scotia and Scotland. Br. J. Obstetr. Gyn. 1998;18(suppl):39.

    Not in our library

34. 34.

    Rothstein H, Sutton AJ, Borenstein M. Publication Bias in Meta-analysis: Prevention, Assessment and Adjustments. 2005.

    Not in our library

35. 35.

    Schneeweiss S, Glynn RJ, Tsai EH, Avorn J, Solomon DH. Adjusting for unmeasured confounders in pharmacoepidemiologic claims data using external information. Epidemiology. 2005;16:17-24.

    Not in our library10.1097/01.ede.0000147164.11879.b5

36. 36.

    Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. J. Am. Med. Ass. 1995;273:408-412.

    Not in our library10.1001/jama.273.5.408

37. 37.

    Smith TC, Spiegelhalter DJ, Thomas A. Bayesian approaches to random-effects meta-analysis: a comparative study. Statist. Med. 1995;14:2685-2699.

    Not in our library10.1002/sim.4780142408

38. 38.

    Spiegelhalter DJ, Best NG. Bayesian approaches to multiple sources of evidence and uncertainty in complex cost-effectiveness modelling. Statist. Med. 2003;22:3687-3709.

    Not in our library10.1002/sim.1586

39. 39.

    Sterne J A C, Welton NJ, Ades AE, Altman DG, Carlin JB. Incorporation of potentially biased evidence in systematic reviews and meta-analyses. Int. J. Epidem. 2008.

    Not in our library

40. 40.

    Taylor BN, Kuyatt CE. Guidelines for evaluating and expressing the uncertainty of NIST measurement results. Technical Note 1297. 1994.

    Not in our library

Cited by papers we hold 7

• Communicating uncertainty about facts, numbers and science · 2019
  No notice
• EXPLICIT: a feasibility study of remote expert elicitation in health technology assessment · 2017
  Not checked
• Methods for network meta-analysis of continuous outcomes using individual patient data: a case study in acupuncture for chronic pain · 2016
  Not checked
• Infliximab, adalimumab and golimumab for treating moderately to severely active ulcerative colitis after the failure of conventional therapy (including a review of TA140 and TA262): clinical effectiveness systematic review and economic model · 2016
  Not checked
• Comparative efficacy of interventions to promote hand hygiene in hospital: systematic review and network meta-analysis · 2015
  No notice
• The Impact of Study Size on Meta-analyses: Examination of Underpowered Studies in Cochrane Reviews · 2013
  No notice
• A Re-Evaluation of Random-Effects Meta-Analysis · 2008
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.