Measuring multimorbidity beyond counting diseases: systematic review of community and population studies and guide to index choice

1Multimorbidity, usually defined as the coexistence of two or more chronic conditions within an individual, is important for patient outcomes and healthcare costs. Because its prevalence is rising as populations age, multimorbidity is attracting increasing attention from the research community worldwide. More than 2800 publications on multimorbidity appeared between 1900 and 2016, 80% of which were published after 2010. As no universally agreed measure or list of diseases exists to define multimorbidity, numerous indices have been developed. These might be designed, for example, to quantify multimorbidity as a covariate in other analyses, for mortality prediction or for risk adjustment. Previous systematic reviews identified multiple indices, but no searches have been done of indices since 2009.

2Multimorbidity research most often refers to a count of chronic conditions. This method does not reflect patients’ experience or the effects of different combinations or severity of diseases. Some indices, however, combine disease counts with severity measures, physiological factors, or demographic items, thereby allowing a more holistic quantification of illness burden.

3The coexistence of both physical and mental illness within multimorbidity is prevalent. A 2018 report identifying priorities for multimorbidity research highlighted the need for more work into this coexistence. Researchers exploring multimorbidity will therefore increasingly need to account for mental disorders. As previous reviews of multimorbidity indices have not covered mental health in depth we identified and summarised all community based multimorbidity measures that include more than simple disease counts, paying particular attention to mental health. This review should help guide clinicians and researchers to select an appropriate index according to their requirements.

4No single accepted term describes the methods of measuring multimorbidity. In this review we use the term “index” to refer to any method of quantifying disease burden or predicting specific outcomes that includes more than a count of conditions. This could be by weighting conditions (for example, by allocating a score to each), adding other elements, or examining other variables such as drug or physiological parameters.

5To capture all relevant publications we conducted a broad search. We included a variety of terms for multimorbidity derived from previous systematic reviews on this topic and other literature discussing terminology in this area of research. We developed the search strategy iteratively with the support of an Academic Support Librarian. The final search terms are listed in the appendix (appendix eTable 1) and include multimorbidity, comorbidity, polypathology, polymorbidity, pluripathology, multi-condition, and multiple chronic conditions. The search was restricted to adults older than 18 years and to English language publications.

6We planned to summarise reports of novel indices and were primarily interested in the original report of each index. Therefore we excluded papers that either used existing indices or measured multimorbidity using only disease counts. In the initial screening process we included only the original form of each scale and not adaptations or updates; these were found later. Records that were not original research papers, such as conference abstracts, letters, and systematic reviews, were excluded. We defined multimorbidity as multiple co-occurring conditions without reference to a specific disease, so excluded papers were those that focused on comorbidities of an index disease or on comorbidities within one disease area (such as the coexistence of several psychiatric conditions). As most people with multimorbidity are adults living in the community and managed in primary care, we excluded articles about children, animals, or people admitted to hospital or living in residential care. We included studies of hospital inpatients when the primary focus was follow-up after discharge (for example, mortality one year later). As resources were limited, we excluded papers when full texts were not available in English.

7On 19 October 2018 we searched Medline, Web of Science Core Collection, Cochrane Library, Embase, PsycINFO, Scopus, and CINAHL Plus from inception onwards.

8Two authors used Covidence software independently to screen titles against exclusion criteria and the subsequent abstracts against the same criteria. We then extracted the full texts of relevant abstracts for further screening. Any disagreements at the title, abstract, and full text stages were resolved by discussion, and a third author mediated unresolved conflicts. We excluded papers that referred to an existing index, but listed the indices that were used when excluding them at the abstract stage. We found the original papers describing these indices and returned them to the title screening stage. Additional relevant titles were found by reviewing previous systematic reviews on this topic, searching the bibliographies of included full text papers, and tracking their citations using Google Scholar. Emerging relevant titles were added to the screening process.

9After the list of included papers had been finalised, we searched their citations on Google Scholar for updates, revisions, or adaptations as well as validation papers. When original indices were adapted and validated numerous times, we listed the original performance and principal adaptations. We did not include adaptations where the original index was translated into another language with no other changes made. To assess the popularity of each index, we took the total number of citations for each original paper from Google Scholar on 7 September 2019, as a proxy for use. We then calculated the number of citations for each whole year since publication. To retain awareness of the context of their initial design and aims, we summarised index updates separately from the original papers.

10We created a data extraction tool containing specific elements of interest for each original index. This tool took account of previous reviews on this topic as well as additional information relating to mental health. We used a broad definition of mental health, comprising any mental disorder, including mood disorders, dementia, delirium, and addictions as well as relevant symptoms. Many papers describe validation of the indices used so details on the size and demographic distribution of the populations under test was important.

11Two authors independently extracted data from all full texts. We compared the consistency of extracted items and resolved any differences by discussion and reference to the original paper, with a third author available in case of substantially differing data extraction.

12The variables of interest during data extraction were first author, year of publication, and name of index; original purpose of the index; characteristics of the population under test, including type of data source (eg, cohort study), location, number of participants, sex and age distribution, and mean number of concurrent medical conditions (when given); components included in the index; weighting method (if any) and details of model for its development; outcome measures; information and resources required to apply the index; internal validation or comparison with another index (if applicable); external validation and performance compared with another index (if applicable); and inclusion of mental health (either in comorbidities or as outcomes).

13We assessed the risk of bias of study design and reporting and aimed to develop overall recommendations for index choice. The Cochrane Collaboration advises against scales that generate total numerical scores, preferring emphasis on individual papers’ performance on each criterion. After our search date the Prediction model Risk Of Bias ASsessment Tool (PROBAST) was published. It focuses on risk of bias and applicability in prediction model studies. As our search was not restricted to prediction models, it was not appropriate to apply this tool to every paper. We therefore developed our own list of criteria having referred to resources available for assessing clinical prediction indices. Our assessment aimed to deal with risk of selection, observer, and funding bias. The list contained 10 questions on the population tested, description of the index, statistical methods, validity, and funding. The assessment tool is available in the appendix (appendix eTable 2), including division into domains. We also included an overall impression of the papers’ risk of bias based on the Scottish Intercollegiate Guidelines Network standard, which was scored separately to the criteria rather than in an additive fashion. Two reviewers independently assessed each paper and resolved disagreements by discussion.

14When choosing an index, its predictive ability and its use elsewhere are important. We generated overall recommendations taking into account the generalisability of participants, selection and clinical relevance of index components, outcome measurement, risk of reporting bias, and model evaluation. These were separated into three main categories: recommended, potentially useful (usually when indices were applicable to certain situations), and not recommended.

15We anticipated finding a wide variety of indices covering diverse outcome measures and therefore planned to summarise these narratively. Because of the range of outcomes included we did not expect to be able to perform meta-analysis. We listed performance and validity statistics as reported by the original papers or validation studies alongside each other, for comparison.

16We did not expect to find indices specifically designed for measuring physical multimorbidity in relation to any aspect of mental health. Therefore, for separate narrative synthesis we planned to seek those indices that mentioned mental illnesses or symptoms, either as comorbidities or as outcomes.

17An early draft of this paper was discussed with a lay contributor who has personal experience of multimorbidity. We incorporated her comments into the text—for example, noting in the introduction that the number of conditions a person has might not reflect their experience of health, and in the discussion her suggestions about outcomes that could be important to patients. The lay contributor also commented on a lay summary of the paper (see appendix page 43), which we amended accordingly.

18The searches yielded 7128 results. A search of bibliographies and citations identified 48 additional relevant titles, and a further 15 titles were added from the list of indices mentioned in excluded abstracts. The total number of titles was therefore 7191. Duplicates were removed using EndNote X8 and Covidence software, leaving 5560 unique records for screening (fig 1). Overall, 5236 titles were excluded at the screening stage, leaving 324 abstracts for eligibility assessment. Of these, 86 full texts were assessed and 35 papers finally included.

19Twenty articles originated from the United States; three from Australia; two each from the United Kingdom, Taiwan, and Italy; and one each from Canada, Spain, Germany, New Zealand, Norway, and India. They were published between 1968 and 2017, with 15 (43%) published since the last systematic review on this topic in 2009. The mean number of participants included in the derivation populations of indices developed after 2009 was 356 906, compared with 75 491 before 2009. The newer indices primarily required access to medical records in 11 (73%) cases, and the remainder (4, 27%) self-report; 10 (50%) indices before 2009 primarily used medical records and 10 (50%) used self-report.

20The majority of papers described one final multimorbidity index, even if they tested various models in development, and four papers concluded with more than one index or measure. For consistency, when articles were summarised and their quality assessed we considered each paper as a whole and noted when more than one index existed. We did not comment on models that used only unweighted disease counts, in keeping with our overall exclusion criteria.

21Four indices primarily used weighted drug counts to quantify multimorbidity, five used diagnostic groups or clusters, and 25 included counts of diseases. Of these, 21 were weighted and nine incorporated other parameters, demographic or otherwise. Four papers used a combination of weighting and additional variables. One index used physiological measurements to diagnose multimorbidity. When diagnoses were required, 14 indices took these from medical records and 15 from self-report. Figure 2 is a spider diagram of the papers, displayed according to their broad category of index components.

22The number of outcomes assessed in each paper ranged from none to seven. Eighteen studies measured mortality as an outcome, 13 aimed to predict hospital admissions, 10 measured general healthcare use, seven measured independence with daily activities or disability (with or without physical functioning), seven measured health related quality of life, five measured overall self-reported health, five measured healthcare costs, and four measured drug use. Mental health was a specific outcome in three papers, with a further six including mental health aspects of established tools (eg, the mental component score of SF-36). Adherence to screening programmes, specific physiological parameters, and care quality indicators were assessed by one tool each. Ten (29%) papers used cross sectional data to derive their index weighting but measured longitudinal outcomes. Table 1 lists the papers according to their original outcomes and index components.

23The applicability of each index depends on study design and intended usage. Table 1 summarises index components and outcome variables. Most of the original papers (27, 77%) contained sufficient information for readers to use the index, usually with lists of included conditions with or without weighting. Of the remainder, access to additional free resources was needed in four, information from proprietary scales was needed in one, and two were missing information that would allow the index to be applied. One index was only available as proprietary software.

24Some of the indices, although designed to measure multimorbidity, were developed in cohorts of people with a specific disease, and therefore this condition was not included in the list of comorbidities. Most indices were based on diagnoses or drugs, but two indices required results of laboratory or other investigations.

25The provenance of papers will affect their applicability to other settings. For example, the majority of papers came from the US where the predominant health system is commercial and healthcare costs are of interest to insurers. Some of the indices were designed with a particular population in mind, such as the questionnaire for Indian primary care. This included diseases that are less prevalent in other geographical areas, such as filariasis and tuberculosis, which might limit generalisability. Other original indices have domains that might be outdated. One example is the Charlson comorbidity index. This index assigns the maximum weight of 6 points to AIDS, but the life expectancy for HIV/AIDS in high income countries has changed considerably since the index’s publication in 1987.

26Although several papers mentioned outcomes as relevant to patient experience, only one clearly described involving patients in the study design, by developing their rating scale with focus groups.

27Appendix eTables 3 to 6 summarise all included papers according to their index’s original purpose, components and outcome variables, and information used. The data source, location, and demographics of the population used to derive or test the measure are listed as they are relevant for context. Our overall recommendations are also included. In appendix eTable 3, the index components are weighted condition counts, in appendix eTable 4, the index components are conditions with additional parameters, in appendix eTable 5 they are weighted drug counts, and appendix eTable 6 comprises the remainder, including diagnostic groups and physiological measures.

28The majority of indices (n=29, 83%) included some form of component weighting. Conditions, diagnostic categories, and drugs were weighted by severity or symptoms, either self-reported or defined by clinicians, or according to their associated outcomes in a derivation cohort. Different methods were used for developing each index, and disparities existed in the level of methodological detail reported. The appendix eTable 7 summarises the method for developing each model, the details provided, and baseline outcome reporting.

29Twenty nine (83%) of the papers contained some measure of mental health or dementia, with 18 of these including mental health markers exclusively as comorbidities (including psychotropic drugs when relevant) and three including mental health markers as an outcome only. Seven measures included different aspects of mental health as both comorbidities and outcomes, and one paper included anxiety and depression as both a comorbidity and an outcome. In appendix eTable 8, we summarise whether each index dealt with aspects of mental health, as either comorbidities or outcomes, and how these were measured. Where papers discussed specific findings relating to multimorbidity and mental health, we present their conclusions.

30Using our quality assessment tool, we classified six papers as high quality with little or no risk of bias in reporting and seven as low quality with moderate to high risk of bias. The remaining 22 papers were of satisfactory quality. Of the five domains we assessed, the best reported were index description and funding source. Validity and statistical methods were the least well reported across all papers. Table 2 shows the scores for all papers across each domain. As we had agreed in advance to judge the overall impression without summing domain scores, the domain scores did not always lead to the same overall impression. For example, one study was given an overall impression of satisfactory with a total domain score of 6, whereas another study scored 8 and was also deemed satisfactory.

31It was not possible to quantify publication bias owing to the variety of methods and outcomes used. It is likely that more unpublished methods of measuring multimorbidity exist and are used in clinical practice, especially tailored to specific patient populations or available clinical information.

32As a proxy for usage, we calculated the number of annual citations for each paper. The number of citations for each year since publication ranged from three to 949, with a median of 8.8 (interquartile range 5.3-16.2). This information is listed alongside measures of the indices’ performance at predicting outcomes and validation in appendix eTables 9 (indices without external validation) and 10 (externally validated indices). Sixteen original papers described designing indices within a derivation cohort and testing their ability to predict specific outcomes within a separate validation set.

33Fourteen original papers measured an index’s performance at predicting outcomes and 20 compared an index to an existing measure of multimorbidity. Fourteen indices were validated elsewhere, of which 11 were compared with other indices and three only measured ability to predict outcomes. Among the indices that were externally validated, 11 were tested at predicting different or additional outcomes to those in the original index design. Some indices were tested against other indices that had been developed with different original outcomes—for example, the Charlson index where the outcome in question was admission to hospital or health related quality of life.

34Thirteen (37%) of the indices had updates or adaptations published, by either original or separate research teams. These revised versions included updated comorbidities or weights, focused on specific patient groups, or mapped a clinical index to codes for administrative data. Two of the indices are risk adjustment methods undergoing regular review and updates. The relevant indices are listed in appendix eTable 11 alongside a summary of their adaptations and updates and any reported performance metrics. The older and widely used indices such as Charlson, Chronic Disease Score, and Cumulative Illness Rating Scale have been adapted and updated many times; we include the most cited versions. Most updated indices are broadly based on the aim and outcome measures of the original, with some exceptions.

35Of the indices that were not updated, in some cases this was because the original index was unsuccessful at predicting specific outcomes or was not designed for use outside of the original study.

36In this review we collated descriptions of 35 distinct multimorbidity indices. The papers were diverse in study design, intended purpose, and variables included. Similarities did, however, emerge, such as index components concentrating on conditions, diagnostic categories, drug classes, or physiological measures. Mortality was the most commonly studied outcome, with healthcare use, hospital admission, functional ability, and health related quality of life as other important groups. Those that measure mortality will be of most relevance to clinicians and researchers, whereas healthcare use and costs are more useful to healthcare providers and funders, particularly in predominantly private healthcare systems. For patients, the most relevant outcomes might be quality of life or self-reported health.

37A major strength of this review was the use of an updated search in a rapidly expanding area of research and a focus on multimorbidity measures that specifically include mental health.

38Although the medical subject heading (MeSH) term “comorbidity” has existed since 1990, a new MeSH term, “multimorbidity,” was introduced in January 2018, after our search had been designed and pre-registered. We found that the word “indexes” was used in some titles when we had used “indices” in our search terms. One paper was found by citation tracking and had apparently been missed during our search because we had omitted the term “score.” However, we aimed to minimise the number of missed relevant papers by searching bibliographies, citations, and indices that had been mentioned in the abstracts we excluded. A limitation of this review is that we limited our search to full texts in the medical literature, excluding conference abstracts and other grey literature. This approach might have missed indices in clinical use that are either unpublished or based on guidelines from healthcare quality institutions.

39In this review, we excluded papers that used simple counts of conditions to measure multimorbidity, focusing instead on indices. We chose to make this distinction because indices tend to include multiple parameters to quantify different aspects of multimorbidity and use sophisticated models to predict outcomes. However, as disease counts are the most commonly used method of measuring multimorbidity, their exclusion is a limitation of our work. The ease of applying disease counts means they are frequently used and they are comparable between studies as long as the list of candidate conditions is clear. One paper in this review reported that a count of diagnosis clusters was a better predictor of healthcare expenditure than more complex indices. Other research has drawn similar conclusions. Reviewing the use of disease counts is outside the scope of this paper and has been discussed elsewhere. Simple counts of drugs have also been shown to predict healthcare use and mortality, and using these or disease counts are more practical than indices in many settings. For example, they are used in clinical care, as they do not require calculations or particular software, or in large population studies where the impact of each condition on individuals is unknown.

40As this review is aimed at those undertaking population or community research, we also excluded studies of people who had been admitted to hospital or live in residential care. This meant that several commonly used indices were not included in this review, such as the Elixhauser index. We did include the Charlson, PROFUND, Self-Administered Comorbidity Questionnaire, and High-Risk Diagnoses for the Elderly Scale indices as although the studies recruited hospital patients, they did so when the patients were discharged, and the main period of interest was later, in the community setting.

41Some of the indices have been in use for many years and have several adaptations. The Charlson index is the most widely known and warranted its own systematic review within a medical specialty (critical care); we have only briefly summarised its performance and the principal adaptations. More information is available in another systematic review on this topic.

42Some of the indices were specifically designed for use with administrative data. These might have scored more poorly on quality assessment as our tool focused on reporting and clinical applicability. Our search also included papers that compared different measurements of multimorbidity but that were not intended for clinical use. We included these papers for completeness when they met our inclusion criteria. We aimed to find measures of multimorbidity, and our exclusion criteria included studies of comorbidity with a specific index disease. However, in two of the papers studied the indices had been developed in populations with one condition only, either hypertension or type 2 diabetes. We referred to our protocol and included these studies because their aim was to study multimorbidity rather than comorbidities of those conditions as index diseases. These papers are, however, less generalisable to the general population.

43Our search was intentionally broad, using a wide range of search terms in multiple medical research databases. We included a variety of studies measuring multimorbidity from different perspectives, which is a strength over previous more specific reviews.

44We generated overall impressions of the risk of reporting bias and recommendations for index use, to provide a guide for researchers when choosing an index. Samples in the included studies were, however, heterogeneous, and the indices had varied purposes and components. Therefore, our recommendations are subjective. Formally comparing the predictive ability of the indices is outside the scope of this work but has been comprehensively performed by other investigators.

45The two most recent similar systematic reviews to ours did not formally assess the quality of publications. Fifteen (43%) of the indices we included were published after 2009 and therefore would not have been found by the searches in these previous reviews. This is out of proportion to the increase in multimorbidity publications since 2010, suggesting that many recent papers have used either older measures or disease counts. A newer systematic review on this topic focused only on tools used on administrative data and conducted its search in 2012. A systematic review of multimorbidity systematic reviews, published in 2018, focused on definitions and measurement. However, it did not include a search for new indices, thereby also omitting the 15 papers published since 2010.

46We suggest that to select an index for clinical or research use, clinicians or researchers should first consider their desired outcomes and the information available. Box 1 summarises the process of selecting an index using this review.

47Identify reasons for including multimorbidity—for example, is multimorbidity important because of its association with quality of life or mortality? This will inform which outcomes of original indices are relevant

48Identify the exposure variables available (eg, diagnoses, drugs)

49Identify the outcomes to be measured

50Use figure 3 to choose a recommended index

51Use the information in the appendix (eTables 9 and 10) to compare usage and performance of any suitable indices

52Appendix eTables 3-6 list our overall recommendations, divided broadly into “not recommended,” “potentially useful,” and “recommended,” and figure 3 displays the 12 indices that we recommend according to their design. The 10 indices that are not recommended could be useful for other purposes, such as recording symptoms or comparing models, but our recommendations focus on those that are practical for designing multimorbidity research.

53At least 35 objective measures of multimorbidity are available for people living in the community, and each of these uses different variables to generate a score or index, linked with various or no outcome measures. We found no specific index for investigating the interplay between mental and physical multimorbidity, although this issue is dealt with in a variety of ways across the measures. The array of index components and outcomes means that a validated measure exists for many applications, including clinical, research, and cost prediction. It is important when choosing an index to consider its original purpose and the outcomes for which it is validated. Given the differing methodologies of multimorbidity research, it would not be appropriate to assume that a single index could definitively measure multimorbidity in all settings. However, with this research area at risk of saturation, we propose that anyone measuring multimorbidity should study existing indices before developing new ones.

54It is common for people to have two or more co-occurring chronic conditions (multimorbidity)

55Researchers and clinicians use many different indices to measure multimorbidity

56At least 35 objective measures of multimorbidity are available for people living in the community and each of these uses different variables to generate a score or index, linked with various or no outcome measures

57No specific index investigates the interplay between mental and physical multimorbidity, although this is dealt with in a variety of ways across the measures

58The recommendations in this study should guide researchers to find a suitable index for their purposes