Prevalence and Estimated Economic Burden of Substandard and Falsified Medicines in Low- and Middle-Income Countries

1Increasing access to essential medicines is integral to the effort to reduce global morbidity and mortality. While access and health outcomes have generally improved in recent decades, these efforts face a serious obstacle from the threat of substandard and falsified (SF) medicines.,, Poor-quality medicines increase risks of morbidity and mortality by prolonging illnesses and heighten the risk of treatment failure, poisoning, and adverse drug interactions., Circulation of SF medicines with little clinical effectiveness also places entire communities at risk of drug resistance, posing a threat to global treatment effectiveness, as well as undermining people’s overall trust in the health system and its legitimate health care professionals.,,,, Over time, diminished trust in licensed health care professionals may further encourage informal care-seeking and self-medication. Poor health outcomes can also erode trust in the manufacturers of genuine pharmaceutical products, which are often challenging to distinguish from SF ones without the use of verification technologies.

2The risks posed by these medicines extend beyond health outcomes. Poor-quality medicines cause increased costs for patients and the health system., Some of these costs, such as resources wasted on ineffective therapies and treating additional complications, are borne primarily by consumers and health facilities., Others, such as decreased economic productivity resulting from prolonged illness, reduced sales and tax revenue, and costs of anticounterfeiting initiatives, are borne by governments, companies, the pharmaceutical industry, donors, and society as a whole.,, Furthermore, SF medicines purchased through the use of personal savings, sale of assets, or borrowing can lead people into poverty.

3Substandard and falsified medicines are a complex but critical global health issue.,,, The World Health Organization (WHO) estimates that 10.5% of medicines worldwide are substandard or falsified. Furthermore, most of the burden falls on low- and middle-income countries (LMICs) because of poor pharmaceutical governance, weak technical capacity, and poor supply-chain management., Until recently, the efforts to combat SF medicines have been fragmented because of the complexity of the issue and intellectual property rights disputes. In 2017, in an effort to draw the issue away from intellectual property rights and reframe it solely as a public health issue, the World Health Assembly officially adopted the term substandard and falsified to replace the previous term, substandard/spurious/falsely labeled/falsified/counterfeit (SSFFC). Substandard medicines are officially defined as “authorized medical products that fail to meet either their quality standards or specification, or both” and may result through poor manufacturing, shipping or storage conditions, or when the drug is sold beyond the expiration date. Falsified medicines are defined as “medical products that deliberately/fraudulently misrepresent their identity, composition or source.”

4This issue not only has significant health and economic consequences, but directly threatens global health security and efforts to meet the United Nations Sustainable Development Goal 3.8, to achieve universal access to safe and effective essential medicines., Despite this, the full extent of the problem is unclear., Furthermore, research efforts to examine the issue have often faced poor methodological quality and exhibited high amounts of variability. To address these issues, we systematically reviewed and analyzed the existing evidence to assess the prevalence and estimated economic burden of SF essential medicines across LMICs.

5For this systematic review and meta-analysis, 5 databases were searched: PubMed, EconLit, Global Health, Embase, and Scopus. A research librarian was consulted to aid in the creation of 2 separate searches and ensure all relevant studies were identified. The first search used terms related to “substandard and falsified medicines” (incorporating SSFFC terms) and the second, “quality of medicine.” Both searches were run with terms associated with “low- and middle-income countries.” Additional articles were incorporated through systematic searches of the WHO Essential Medicines and Health Products Information Portal, the United States Pharmacopeial Convention Medicines Quality Database, publications in the Worldwide Antimalarial Resistance Network database, and references of included articles and pertinent literature reviews.,,,,,,,,,,, The comprehensive search strategy and terminology as well as a list of additional sources searched are presented in eAppendix 1 in the Supplement. This systematic review and meta-analysis is registered in the PROSPERO database and is reported according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guidelines. Institutional review board approval was not required as this study did not involve human subjects research.

6The primary eligibility criteria for this systematic review were that the article examined the quality of essential medicines, the prevalence of SF medicines, and/or the economic impact of SF medicines. Economic impact is defined broadly as any economic estimate associated with the manufacture, trade, sale, or consumption of SF medicines. Peer-reviewed articles in English, Chinese, French, Portuguese, Spanish, or German published before November 3, 2017, were included in the review. All study locations not in LMICs, as classified as the World Bank at the time of review, were excluded. Abstract-only publications, correspondence without data, studies of medicines that are not classified as essential medicines by the WHO, case reports, and publications that did not include sample sizes of tested medicines were also excluded.

7All unique articles were independently assessed by 3 of us (D.G.H., S.B., and D.R.E.) based on title and abstract. Those marked for inclusion, or whose title and abstract were not sufficient to determine inclusion, were then reviewed using the full text. Any discrepancies between reviewers were resolved by a third party (S.O.). Full-text records were sought and obtained through the library systems of 3 major US research universities. Selected articles were then categorized into 1 or both of 2 groups: (1) primary prevalence studies and/or (2) articles containing an estimate of economic impact.

8Four of us (D.G.H., S.B., S.K.L., and D.R.E.) independently conducted the data extraction with oversight by a third party (S.O.). Data describing prevalence, type of medicines, country setting, and sample size were extracted from all included primary prevalence studies. Studies were grouped according to the following categories: type of medicines tested, sample size, continent, and year of publication.

9Studies including an estimate of economic impact were compiled separately and data were extracted on estimate source, period of analysis, geographic scope, target medicine, and estimated economic impact. Citation mining was conducted to determine the root sources of the cited statistics. For every estimate of economic impact, we noted the root source, type of literature, and methods used, or recorded that the root source was untraceable if it could not be traced back.

10The meta-analysis of the prevalence of SF medicines focused on studies that tested 50 samples or more and adequately reported sampling and testing methods to ensure the rigor of prevalence estimates. Studies that did not report primary data, included previously reported data, or included data from regulatory laboratories that only tested suspected medicines were excluded from the meta-analysis. Where available, uncertainty ranges or minimum to maximum prevalence were noted alongside summary statistics. Mean prevalence across studies—weighted by sample size and Medicine Quality Assessment Reporting Guidelines (MEDQUARG) score using a random-effects model—was determined overall and by world region. Mean weighted prevalence was also assessed across subcategorizations of 8 study characteristics: sample size, publication year, sampling method, purchasing method, chemical analysis, chemical testing method, conflict of interest, and MEDQUARG score. The prevalence for each subcategorization was compared with the overall prevalence with a 1-sample t test and with the subgroup gold standard by an unequal variances t test. Overall uncertainty ranges were estimated using a 95% confidence interval based on the calculated standard deviation.

11Additional data were extracted for these studies in the meta-analysis, including methods of sampling, purchasing, and pharmaceutical quality analysis. Sampling methods were categorized as random, convenience, survey-based, or not specified; purchasing method was categorized by use of mystery clients, using overt methods, or not specified; and pharmaceutical quality analysis was categorized by the specific chemical analyses. Additionally, each study was assessed to determine whether it (1) examined the actual presence of active pharmaceutical ingredients (APIs), (2) assessed the quantity or percentage of the APIs, and/or (3) looked for other ingredients (excipients and other analytes). Use of Raman spectrometry, any mention of impurities, or highlighting unknown peaks in chromatographs were counted as searching for other ingredients.

12To assess the quality of the studies included in the meta-analysis, studies were independently examined by 2 of us (D.R.E. and S.K.L.) according to MEDQUARG. Each study was assigned a 12-point MEDQUARG score adapted by Almuzaini and colleagues with scores of 6 or greater being considered acceptable quality. The interrater reliability was assessed between the 2 reviewers and, where possible, the original Almuzaini scores using the Spearman ρ. Additional information about the MEDQUARG scoring metric, this study’s reported MEDQUARG scores (eTable 1 in the Supplement), and results of the interrater reliability assessments are presented in eAppendix 2 in the Supplement. Additionally, studies included in the meta-analysis were analyzed for potential conflict of interest by examining the articles’ author statements, funding source, and/or institutional affiliations for a direct partisan or industry relationship. Studies with a description of funding source with no apparent partisan or industry affiliation were considered free of potential conflict of interest. Other studies that did not include a funding source and had no discernable industry author affiliations were categorized as unclear.

13Study heterogeneity was evaluated using a random-effects model and reported using the Cochran Q and I2. Publication bias was evaluated using a funnel plot analysis (eFigure 1 in the Supplement) with a regression test for funnel plot asymmetry. Baujaut and influence plot analyses (eFigures 2 and 3 in the Supplement) were conducted to examine which articles contributed the most heterogeneity. A mixed-effects model was used to test for potential modifiers. The results of these analyses are included in eAppendix 3 in the Supplement. All analyses were made using R statistical software version 3.3.2 (R Project for Statistical Computing).

14Our searches yielded a total of 4284 citations, of which 3164 were unique and screened based on title and abstract. The full text was assessed for 754 articles. We identified 265 primary data collection studies that sought to determine the prevalence of SF essential medicines in LMICs (Figure 1). Studies not included in the meta-analysis are shown in eReferences 1 in the Supplement.

15Globally, data on SF medicines in LMICs came primarily from articles focused on Africa (133 studies [50.2%]) and Asia (90 studies [34.0%]). Eight studies (3.0%) covered countries in South America, while 3 studies (1.1%) tested samples from each of Europe, North America, and Oceania. Twenty-five studies (9.4%) tested samples from multiple continents. More than half of the 265 prevalence studies (157 [59.2%]) were published in this decade (2010-present) with 63 (23.8%) published in the last 3 years (2015-2017). Antimalarials (117 studies [44.2%]) and antibiotics (104 studies [39.2%]) were the most commonly examined medicines, including 35 studies (13.2%) that investigated both. The total number of samples tested was 400 647, with a median (interquartile range) study sample size of 41 (10-242) (see eTable 2 in the Supplement for additional details on study characteristics).

16Our meta-analysis included a subset of 96 studies,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, (67 839 drug samples) that met inclusion criteria. Figure 2 shows the results of the meta-analysis of the prevalences grouped by region and medication category. The average overall prevalence of SF medicines was 13.6% across LMICs (95% CI, 11.0%-16.3%). Regional prevalence estimates ranged from 18.7% in Africa (95% CI, 12.9%-24.5%) to 13.7% in Asia (95% CI, 8.2%-19.1%) and 14.4% (95% CI, 0%-33.2%) for other single-region studies. Studies that tested samples across multiple continents observed a lower average prevalence, with a pooled prevalence of 11.6% (95% CI, 5.8%-17.5%). The average prevalence of SF medicines was 19.1% (95% CI, 15.0%-23.3%) for antimalarials and 12.4% (95% CI, 7.1%-17.7%) for antibiotics. Table 1 presents the data extracted from these studies grouped by medication class. A map of the calculated national prevalence of SF medicines is presented in Figure 3 (eFigures 4-7 in the Supplement present the reported prevalence of SF medicines by each study in the meta-analysis grouped by region).

17Table 2 presents the summary statistics and weighted prevalence of the subcategorizations of 8 study characteristics in the meta-analysis. Statistical analysis of the association between the mean SF prevalence of each subcategorization and the overall prevalence (13.6%) found studies with smaller sample sizes to report significantly higher mean prevalence (sample size 50-99: mean prevalence, 34.4%; P =.001 and sample size 100-249: mean prevalence, 31%; P <.001). Conversely, the prevalence of studies that used convenience sampling (7.1%; P =.001) or survey sampling (2.3%; P =.009) and those that analyzed samples with the Global Pharma Health Fund’s Minilab, a mobile minilaboratory suitcase, were found to be significantly lower (7.7%; P <.001) than the overall prevalence of 13.6%.

18To further examine the association between methodological quality of the studies and the reported SF prevalence, we statistically compared the mean prevalence of SF medicines for each of the 8 subcategorizations with the gold standard for each study characteristic. Small sample sizes and less rigorous sampling methods resulted in significant differences from the categorical gold standard of larger sample sizes (7.0%) and randomized sampling (17.4%). The mean prevalence of studies with potential conflict of interest (9.3%) was also significantly lower than both the overall prevalence (13.6%; P =.01) and studies without discernable conflict of interest (14.2%; P =.03). Interestingly, the mean SF prevalence of studies with adequate reporting quality (MEDQUARG score ≥6; 15.9%) was 5% higher than studies with a lower quality score (10.9%) but not significantly so (P =.07).

19A random-effects model was used to examine studies for heterogeneity and publication bias. Studies in the meta-analysis indicate large amounts of heterogeneity (I2 = 99.9%), and the regression test for funnel plot asymmetry indicates publication bias (P <.001). A mixed-effects model was used to test for 8 different potential modifiers where sample size was found to be the only significant modifier (P =.04) (eAppendix 3 in the Supplement).

20Our search for economic impact estimates yielded 65 unique records with each citing 1 or more of 17 different estimates of the economic effect of SF medicines. Three of these estimates were found to be citation errors, 4 did not estimate total economic impact or market size, and the root sources of 2 were untraceable, resulting in 8 economic estimates.

21eTable 3 in the Supplement presents the original sources of the 8 economic estimates, primarily of market size, ranging from $10 billion to $200 billion annually (median, $31.25 billion). Of the 8 estimates traced back to their origin,,,,,,,, many were old and based on crude calculations by authors without methodological disclosure. Four estimates came from reports from international or intergovernmental organizations but 3 did not note how the estimates were derived. Three estimates were introduced as a rough back-of-the-envelope calculation in a peer-reviewed journal and 2 sources were potentially partisan involving a moderator’s guide from a think tank and a book.,

22Findings from this systematic review and meta-analysis reveal that 13.6% of essential medicines tested in LMICs failed quality analysis. The highest prevalence of poor-quality medicines was observed in Africa, where 18.7% of samples were substandard or falsified. Deficiencies in quality were more prevalent among antimalarials (19.1%) than antibiotics (12.4%), while further studies are needed to understand the quality profile of other essential medicines.

23These findings are similar to the estimated prevalence from a 2017 report by the WHO and consistent with ranges reported in other reviews.,,,,,,,,,, All studies within these reviews that met our inclusion criteria were included in our review; the slight variance between our results and those of previous analyses is likely due to these reviews including studies with small sample sizes as well as the sheer amount of study heterogeneity.

24This review identified a significant amount of study heterogeneity and potential issues of quality of the prevalence data reported in publications. This is a significant issue as poor-quality prevalence data affect analyses of the health and economic effects of SF medicines as well as crucial policy and regulatory efforts to address the issue. Unfortunately, controversy over the role of industry in defining the problem of SF medicines has slowed global efforts to address this issue in recent decades., The International Medical Products Anti-counterfeiting Taskforce (IMPACT), established by the WHO in 2006, disbanded over perceived industry connections, and intellectual property concerns waylaid the agreement on the definition of SF medical products. Concerns that anticounterfeiting efforts could adversely affect the legitimate generic drug industry—essential for access to medicines in LMICs—have been central to the debate. This dispute has distracted from the public health and socioeconomic consequences as well as efforts to accurately assess the scope of the issue. As a result, efforts have been disjointed, and numerous studies with small sample sizes that do not use rigorous randomization, collection, and analysis methods have been conducted.

25This review also identifies a significant gap in the literature on the economic burden of SF medicines due to the poor-quality of reported economic estimates and limited focus, primarily on market size. Robust economic analysis capturing the broader economic burden of SF medicines, such as additional costs of treatment and productivity losses, is critical to understanding the extent of the problem, raising awareness, developing intervention strategies, and fostering change. Future economic research is important to inform efforts to combat the falsification of medications and should be conducted following rigorous economic methods.,

26As demonstrated in the results of our prevalence subgroup analysis, use of less rigorous research, analysis, and reporting methods is detrimental to efforts to assess the scope of the issue because of the number of biases they introduce.,, Studies aiming to support policy development should therefore follow rigorous standards of sampling, analysis, reporting, and disclosure. While metrics such as MEDQUARG exist to guide in the reporting of studies of the prevalence of SF medicines, greater effort and emphasis need to be placed on researching and standardizing international sampling, collection, and analysis protocols.,, Improving quality control and laboratory capacity in LMICs is also crucial as the GPHF-Minilab that is used in these settings in lieu of full pharmacopeial analysis has poor sensitivity to detect substandard medicines.,,, Furthermore, to ensure that these efforts bear fruit, greater transparency is needed in the disclosure of industry-related potential conflicts of interest. This heterogeneity in findings serves to caution against extrapolating SF prevalences to other regions and across medicine categories.

27Countries with weak pharmaceutical governance and poor pharmacovigilance are at the greatest risk from SF medications.,, Weak regulatory capacity to license manufacturers, ensure good manufacturing practices, and perform quality control encourages the illicit manufacture and distribution of SF medications. Poor supply chain management and surveillance not only open the door to allow SF medicines to permeate the supply chain, but also cause stock-outs that drive patients to purchase medicines from unregulated markets. Therefore, efforts to improve supply-chain management, surveillance, and regulatory capacity in LMICs are essential to reduce the threat of SF medicines. The successes of the Promoting the Quality of Medicines and the WHO prequalified drug programs demonstrate that these efforts can reduce the prevalence of SF medicines.

28While the effects of SF medications disproportionately rest on LMICs, SF medicines originate from and are reported in every country worldwide., The global nature of the medicine supply chain implies that weaknesses in any country in the supply chain affect all the countries downstream.,, This threatens global health security by increasing transmission, morbidity, mortality, and antimicrobial resistance, highlighting the need for a unified global effort to address the issue.,, In 2013, the WHO Global Surveillance and Monitoring System was launched to gather data, improve reporting, and strengthen regulatory capacity globally. While this is an important first step, additional efforts to implement laws on drug quality and improve data sharing among stakeholders are essential to complement research and technical initiatives. Nayyar et al have called for an international convention on SF medicines, similar to the 1929 treaty that internationally criminalized counterfeit banknotes or the Framework Convention on Tobacco Control., Such a convention could (1) accurately define SF medicines, (2) require signatory countries to enact national laws (and consequent prosecution penalties) criminalizing intentional manufacture, trafficking, or selling of SF medicines, (3) provide a legal and institutional framework on convergent medicine regulation, and (4) provide LMICs with financial and technical assistance to effectively join local and regional SF medicine regulatory networks. Without such concerted global effort, the global supply chain, and thereby every country within it, remains vulnerable to SF medicines.

29Systematic reviews are inherently limited by their search strategies, databases searched, and the inclusion and exclusion criteria selected. To address this limitation, we ran 2 searches, performed a systematic reference review, and examined other pertinent database sources. Furthermore, as with all meta-analyses, ours is limited by the quality of the included studies and any biases they may contain. As demonstrated by the high amount of heterogeneity, the summary statistics reported in this study reflect a wide range of studies and methods, and as a result are subject to various limitations. For studies involving multiple medicines and countries, only the total sample size was included in the prevalence calculations. It is therefore possible that the regional variation observed is explained by differences in sample size by medicine, or that the variation by medicines is explained by the geographic distribution of samples. Differences in study quality, sampling, or purchasing and collection method across regions and medicine categories may have also introduced bias. To control for these potential sources of bias, we selected studies that tested 50 or more samples and removed studies with very poor sampling methods or no description of study methods. To further ensure the rigor of our reported prevalence estimates, each study prevalence was weighted by sample size and a metric that assessed the quality of the reported studies. Quality analyses indicate that there is significant publication bias as well as a moderating effect of sample size, even after controlling for it, which demonstrates the significance of rigorous sampling methods. Finally, reported economic impact estimates are limited by the poor quality and large heterogeneity of available data. Despite these limitations, we believe we appropriately controlled for bias to the best of our ability and have identified and synthesized articles in a systematic and methodical manner.

30Our findings suggest that SF essential medicines are a substantial and understudied problem in LMICs with high estimated economic impact. Reducing their prevalence is imperative to the Global Health Security Agenda, reaching the United Nations Sustainable Development Goals, and global efforts to curb antimicrobial resistance., Although the literature on prevalence continues to grow, methodological standards are needed to improve generalizability and facilitate comparison across studies. Precise, independent estimates are also needed to describe the health and economic effects of poor-quality medicines to build the evidence base for successful policy interventions to curb SF medicines in LMICs. Efforts to strengthen supply-chain management, surveillance, and regulatory capacity are essential to effectively control SF medicines. Globalization necessitates global coordination across national, regional, and sectoral stakeholders to improve the regulation, standardization, and surveillance of the quality of medicines worldwide.