On what basis are medical cost-effectiveness thresholds set? Clashing opinions and an absence of data: a systematic review

1A prominent issue concerning many national healthcare systems today is how much to invest in new medical products, services, and intervention programs [1]. An integral part of this type of investment regards potential improvements in quality-adjusted life years (QALYs) and how much healthcare systems should be willing to spend for additional QALYs for their patients [1]. The cost-effectiveness (C/E) threshold, a tool used by countries to dictate the maximum financial investment a country or organization is willing to invest to give a patient an additional QALY, ranges greatly from country to country depending on methods and assumptions used [2]. Though not all countries use a formal C/E threshold system, the valuation of a QALY, when adjusted for inflation and purchasing power parity (PPP) to 2015 USD, can range from as little as 4419$ USD per QALY gained in Thailand to 173,971$ USD per QALY gained in Norway [3]. Though some argue that C/E thresholds are arbitrary and perhaps should be abandoned as a formal measure, two opposing approaches argue for the existence of formal, evidence-based C/E thresholds, though they differ in their reasoning and may produce two very different C/E threshold values [2,4]. Explicit cost-effectiveness thresholds act as a hard limit and are the major determinant in the decision-making process [5]. Conversely, implicit thresholds are not necessarily official ranges or values used by decision-makers, but can be inferred retrospectively by analyzing the cost-effectiveness of interventions previously evaluated by decision-makers [5–7]. With implicit thresholds, there exists greater potential for decision-makers to feel increased pressure to approve or reject certain drugs due to the zeitgeist of the current political landscape, regardless of the potential impacts on a population’s health which may cause social or political tension [8].

2Those favouring an extra-welfarist approach argue in favour of using opportunity cost as a method of determining C/E thresholds [5,9,10]. According to this theory, the general public does not have the data or expertise to determine how resources can effectively be allocated to maximize the health of a population and the decision-making process should be reserved for experts. Given the nature of federal expenditures constrained within finite budgets, any direct investment in national healthcare is a draw from a different area; some indirect investments, to areas such as education or waste management, could result in larger increases to the average national level of health than investing directly into the healthcare system. Similarly, internal to healthcare systems, the funding of a new intervention imposes additional costs on restricted healthcare budgets and may require fund reallocation from other interventions and services elsewhere within healthcare systems [10,11].

3Alternatively, C/E thresholds can be based on willingness-to-pay (WTP) per QALY values. This method is based on information about populations’ preferences so that they can be better reflected in the healthcare system. This welfarist approach argues that healthcare is financed through tax systems and thus a population’s preferences should be reflected onto how much value is placed on healthcare services [5,9,12]. Welfarists also argue that populations have the best available knowledge of how they value their own health and thus population preferences should be the basis of defining C/E thresholds (a discussion about WTP per QALY studies can be found in Appendix A).

4Presently, there is no commonly agreed-upon value or method for determining C/E thresholds. Some federal health systems compare their own gross domestic product (GDP) per capita to the cost per QALY of new medical interventions when deciding to approve new drugs based on the World Health Organization’s (WHO) one-to-three times GDP per capita recommendation [13,14]. However, this recommended threshold is based on a long-standing misinterpretation and not on any methodological justification [3,15–20]. Also commonly referenced in American health economic literature is the value of 50,000$ USD per QALY. According to Grosse [21], this value stems from the cost of dialysis in the 1980s. Similarly, 100,000$ USD per QALY is often referenced as the suitable C/E threshold without justification. In US-based cost-utility analyses, 77.5% of all authors use either 50,000$ USD or 100,000$ USD per QALY as a reference point for cost-effectiveness [22].

5According to the Grossman model of health economics [23], investment in healthcare systems faces decreasing marginal return to scale. In line with this model, countries with higher GDPs often have more funds available to invest in healthcare systems and may have a populace more interested in experiencing a higher level of health [23]. This results in rich countries being at particular risk of overinvesting in new, expensive medical interventions. Baker et al. [5] modeled the relationships between expenditure, opportunity cost, WTP, and C/E thresholds in healthcare investment, and illustrated diminishing returns to scale and the potential for an efficient C/E threshold where marginal cost (MC) equals marginal benefit (MB) (illustrated and further explained in Appendix C).

6The aim of this systematic review is to describe prominent approaches to setting a C/E threshold, compile available national level C/E threshold and WTP data, and discern whether there are possible associations between C/E thresholds as well as WTP per QALY and other variables. This review will also discuss obstacles faced due to data limitations.

7A systematic search of the relevant literature, using Google Scholar, PubMed, and the Umeå University Library Catalogue, was conducted using the keywords ‘cost-effectiveness threshold’ and ‘decision making’ and ‘healthcare systems’ and ‘QALY’, published in English between 2010 and 2016 to exclude possible out-of-date data (Figure 1). The search was conducted by two reviewers working independently to create one database resulting in 240 papers. Four additional papers were identified after checking the references and ‘cited by’ sections of the papers identified in our searches. Based on recommendations by colleagues and to ensure that no grey literature was overlooked, the website domains of the Organisation for Economic Co-operation and Development (OECD), the Zorginsituut Nederland, and the Grupo de Ativistas em Tratamentos Portugal were searched using Google Domain search function for the term ‘cost-effectiveness thresholds’.

8In total, 238 papers were screened due to our selection criteria. Studies that only justified the stated C/E value using WHO’s ‘recommendations’ were excluded while studies that stated their value as being ‘the most commonly accepted value for approving drugs in this country’ or cited government data were accepted. A further exclusion criteria pertained to some papers presenting values for countries as a whole, when in reality the country has different C/E thresholds for different provinces/states or for different areas of medicine [3]; these were filtered on a case-by-case basis (explained in Appendix A). From the resulting papers, 17 countries were identified from 9 unique sources to have formal C/E thresholds within our inclusion criteria. The C/E thresholds of England, Thailand, and Ireland are explicit, while the others are implicit.

9A separate search was conducted through Pubmed, Google Scholar, and the Umeå University Library Catalogue for the keywords ‘ “Willingness to pay per QALY” country’. The search was conducted by two reviewers working independently to create one database. Only results published in English between the years 2000 and 2016 were included (Figure 2).

10WTP per QALY studies were filtered using several criteria. It was required that the participants in each study be representative of the general population, the sample size be greater than 100 [24], and the diseases on which the health states are based on should be unknown to the participants (as further described in Appendix A) [25]. From our initial search, six articles were identified as relevant to our research question. An additional six articles were identified through the references of articles found in our initial search. Of these six additional articles, three were determined to be duplicates. This resulted in a total of 9 studies meeting our criteria, comprising WTP per QALY data for 13 countries.

11Data are presented in 2015 US dollars PPP. Historical currency exchange rates, inflation rates, GDP per capita, and Purchasing Power Parity (PPP) were calculated for all retrieved data with the XE Online Historical Currency Converter [26], the US Inflation Calculator [27], and the World Bank Online Database for GDP per capita [28] and PPP conversion [28,29], respectively. Information regarding Taiwan’s PPP and inflation was taken from the CIA world factbook [30], as data were unavailable through the World Bank website.

12Though many explicit thresholds are not frequently updated or adjusted [24], capturing the financial value of the threshold at the time data were published best reflects the values used in the decision-making process that determined the C/E threshold. Health-adjusted life expectancy (HALE) was sourced from the Global Burden of Disease study (2010) [25] and was selected as a measure of national average health due to its ability to compare health between countries and its extensive use in health economic literature. Potential linear correlations of WTP per QALY with HALEs, GDP per capita, and C/E thresholds were investigated (figures in Appendix B). Non-linear (exponential and polynomial) correlations of C/E thresholds with HALEs and GDP per capita were investigated because non-linear relationships were expected [5,31]. All the correlations were calculated with Microsoft® Excel® 2016 (Microsoft, Redmond, Washington, United States).

13C/E thresholds are presented in Table 1; extra notes have been added to point out particular aspects of certain studies. Some countries without publicly available C/E thresholds have published official justifications for not using these measures, while most simply have no publicly available data [32,33].

14The PPP-adjusted C/E thresholds (Table 1), are correlated with HALEs; a quadratic relationship may be seen with a polynomial correlation of 0.633 (R = 0.63) with an apex of approximately 100,000$ USD per QALY (Figure 3).

15A relationship can also be seen between the C/E thresholds and GDP per capita (Figure 4). Most countries with formal C/E thresholds fall within WHO’s ‘recommendation’ of one-to-three times GDP per capita and are from OECD countries. Two additional solid lines have been added to this graph to illustrate which countries have thresholds that fall within WHO’s ‘recommendation’ of one-to-three times GDP. The dotted line expresses a line of best fit with a polynomial correlation (R = 0.84).

16WTP data are amalgamated in Table 2. The average WTP per QALY was found to be $77,509, with a range from 1415$ USD to 123,695$ USD, and a standard error of $15,193. No correlations could be identified between the WTP per QALY of a country and C/E thresholds, GDP per capita, and HALEs. Given the low comparability between WTP studies due to differing methodology, all other results pertaining to WTP are displayed in Appendix B.

17Given that only 17 countries have had data published regarding their C/E threshold, it seems that formal and methodological C/E thresholds are a neglected and non-transparent part of decision-making in many countries. Decision-makers may be averse to basing politically sensitive decisions on a single summary measure alone; issues regarding the validity of cost-effective ratios and QALYs may encourage decision-makers to rely more on their own judgment [42]. Many countries lack a formal explicit threshold and use alternative strategies that result in an implicit C/E threshold. Some countries have specified justifications for abstaining from their use while most do not seem to have any reasoning at all.

18Germany is an example of a country that does not use thresholds [17]. Federal policymakers in Germany assert that C/E thresholds are not compatible with German law and history; however, in recent years decision-makers are slowly introducing some economic evaluation into their decision-making process [17]. Other countries do not give an explanation at all. This may be due to a low priority being given to setting a C/E threshold, difficulties in identifying and presentation C/E thresholds, or a lack of health economic expertise.

19The apex of the curve correlating HALEs and C/E thresholds may be illustrating the opportunity cost and diminishing marginal returns in healthcare investment as discussed by Baker et al. [5]. Coincidentally, the approximate apex of the curve (100,000$ USD per QALY) is the same commonly cited value that is believed to be based on no actual evidence, suggesting that it may potentially be a reasonable reference point for C/E thresholds. Though limited by the number of observations, this figure also illustrates the potential for overly inflated C/E thresholds. Some countries have relatively high C/E thresholds but are experiencing overall lower HALEs. Tertiary variables (GDP per capita, total healthcare expenditure, behavioural factors, etc.) could potentially be confounding these results. As more countries refine and formalize their methods of approving new medical interventions, and more data become available, possible correlations with C/E thresholds may be further examined. Almost all identified C/E national thresholds fall within the WHO guidelines of one-to-three times GDP per capita. Although these guidelines have been shown to be largely arbitrary, they still may be influencing decision-makers in C/E threshold setting or coincidently reflect the results of independent C/E setting processes.

20From the welfarist perspective, the average for WTP per QALY of 77,509$ USD lays between the commonly used C/E thresholds of 50,000$ USD – 100,000$ USD per QALY and is similar to the apex of the curve seen in Figure 3. In theory, a strong argument can be made to base C/E thresholds on population preferences; but, in practice, there are too many methodological problems with WTP per QALY studies to make any meaningful decisions based on the presently available data [21,43]. An opportunity cost based approach is an ideologically promising way of setting a C/E threshold; however, data regarding opportunity cost and calculating the impact of specific programs may be unduly complex [5].

21In the case of decision-makers subscribing solely to a rigid C/E threshold, ethical issues and inequalities may arise [44]. For example, a patient with a base QALY of 0.2 who is expected to improve to 0.4 through an intervention would be evaluated the same as someone with a base QALY of 0.8 who is expected to improve to 1.0. Although these QALY gains are numerically ‘equal’, Bobinac et al. [44] argue that this exemplifies an inequity since one of these patients will value the QALY gain far more than the other. This example suggests that determining C/E thresholds from a purely economical perspective may lead to unwanted inequities and other negative results. While setting C/E thresholds solely based on economic arguments potentially may prevent some inefficiencies, there are clearly other benefits from considering other factors in the decision-making process. C/E thresholds can aid decision-makers when appraising evidence, without being the sole metric. Sweden, for example, also considers ‘the human value principle’ (those with the most pressing medical needs should be prioritized) and ‘the need and solidarity principle’ (respecting the equal value of all human life) in their decision-making process [45].

22Practical examples of conflicting opinions between the public and decision-makers can be found in England and the Netherlands. In England, conflicting approaches regarding the funding of new treatments and the valuation of health have led to high-profile clashes of opinion. A recent example of the conflict between federal C/E thresholds and public opinion took place in England where the National Institute for Health and Clinical Excellence (NICE) recommended the rejection of funding for five drugs because their cost far exceeded a C/E threshold [46]. Consequently, the National Health Service (NHS) was not obliged to compensate for these drugs. This decision resulted in patients suffering from chronic lymphocytic leukaemia and mantle cell lymphoma not having access to medication that could improve their quality of life [47].

23In the Netherlands, the College voor Zorgverzekeraars, a Dutch institution similar to NICE, recommended ending compensation for the medicines used to treat Pompe disease and Fabry disease [8]. Both diseases are treatable, though the medication is relatively expensive, costing up to €700,000 per patient per year. After ensuing societal uproar, the Dutch Minister of Health, Welfare, and Sports decided to renew compensation for these medications [8]. The national turmoil that resulted in the Netherland’s initial decision illustrates how a population’s WTP per QALY may not align with current C/E thresholds. Given the high price of this drug, this example demonstrates the mismatch between opportunity cost in healthcare investment and a population’s WTP per QALY.

24A limitation of this study is the lack of available data and information on C/E thresholds available. This presents difficulties in conducting in-depth analyses on how C/E thresholds influence the average health of a nation. After a systematic review of the literature, as well as incidental personal communication with healthcare experts of countries, we found that most countries do not have publicly available data regarding their drug-approval decision-making process.

25Our data review did not reveal how many countries refrain from the use of formal methodologies for approving new medical interventions and data for some countries may have been missed. Since the present research has a limited number of data points, the inclusion of new, additional data could influence our results. Further research is needed to study the possible associations potential relationships described in our study.

26Few WTP per QALY studies have been conducted at the national level (appropriate data for only 13 countries were found) and many took differing approaches. Due to conflicting methods in data collection and analysis the comparability of the results of these studies is limited. If a large-scale, standardized, international WTP per QALY study is conducted, researchers would be better equipped to analyze whether relationships exist between national average individual valuation of health, national average level of health, national C/E threshold guidelines, and other factors.

27Decision-makers need relevant data and strategies in order to make evidence-based decisions when setting C/E thresholds; however, the quality and quantity of data available regarding national average WTP per QALY, opportunity costs, and C/E thresholds is poor in comparison to the importance of adequate investment in healthcare. Given that large decisions regarding investment in new medical interventions are made without appropriate economic justification, the need for further research and data collection is clearly evident. A potential exponential relationship was observed between C/E thresholds and GDP per person, and a potential quadratic relationship was observed between HALEs and C/E thresholds; these relationships may warrant further study if more data become available. By further studying these relationships, researchers may create useful strategies for determining C/E thresholds and whether WHO’s ‘recommendation’ of one-to-three times GDP or the commonly cited value of 100,000$ USD per QALY have merit.