Accuracy of Death Certificates and Assessment of Factors for Misclassification of Underlying Cause of Death

1Cause of death (COD) data from death certificates are often used in epidemiological studies to estimate mortality rates or risk of death from certain diseases. However, the accuracy and utility of COD data from death certificates are uncertain and often questionable.– For cancer mortality statistics in particular, uncertainty regarding the information on death certificates has been discussed for more than 100 years. For example, in early 1900s, Riechelmann reported differences in the number of cancer cases between autopsy and vital statistics reports, and Wells discussed the degree of this influence on vital statistics. In the late 20th century, Hoel et al reviewed the effect of death certificate error on cancer mortality statistics and found a consistent 18% underestimation of total cancer mortality, with an especially large influence on the elderly population (75 years or older). Since around 2000, site-specific analyses for misclassification have been investigated. For example, Percy et al reported on misclassification in colorectal cancer, finding that colon cancer was over-reported while rectal cancer was underreported on death certificates. Similarly, Yin et al indicated that 82% of misclassified rectal cancer deaths were coded as colon cancer deaths.

2For diseases other than cancer, Cheng et al reported death certificate sensitivity and specificity for diabetes of 34.7% and 98.1%, respectively. In their 30-year study, they also reported cardiovascular disease-related diabetes sensitivity stratified by decade of death and showed a time trend of improved sensitivity that reflected increased recognition of cardiovascular disease risk factors. In Japan, Saito et al reported the validity of death certificates for ischemic heart diseases after the ICD-10 code revision. They compared death certificates and the diagnosis examined by a review of the medical records and/or interviews with physicians and reported that the sensitivity and specificity for ischemic heart disease certified as the cause of death was 86.5% and 64.7%, respectively. Ravakhah compared death certificate diagnoses with autopsy report diagnoses in 223 cases and reported that myocardial infarction was more likely to be unsuspected in women and those with advanced age. Kohn reviewed autopsy findings in 200 persons older than 85 years, indicating that the autopsy data were in strong disagreement with the causes of death listed in the vital statistics and proposing that ‘senescence’ be accepted as a cause of death.

3These studies underscore the difficulty in specifying underlying COD (UCOD), especially among elderly people, who tend to have multiple diseases before death. However, the accuracy of UCODs recorded on the death certificates of elderly people has not yet been comprehensively examined for multiple diseases using consecutive autopsy studies. Here, we evaluated the accuracy of UCODs of elderly people recorded on death certificates compared to a reference standard of autopsy findings.

4Of 532 consecutive autopsies performed at the Tokyo Metropolitan Geriatric Hospital (Tokyo, Japan) between February 2000 and August 2002, 450 (84.6%) were included in the present study. No medico-legal cases were included. The average autopsy rate during this period was 32%. All subjects were registered in the geriatric autopsy database (GEAD) at the Tokyo Metropolitan Geriatric Hospital, which contains clinical information (presence or absence of 26 geriatric diseases, as follows: ischemic heart disease, atrial fibrillation, degenerative valvular diseases, hypertension, aneurysm, arteriosclerosis obliterans, dementia, cerebrovascular disorder, Parkinson’s disease, diabetes mellitus, hyperlipidemia, malnutrition, osteoporosis, degenerative osteoarthritis, aspiration, chronic obstructive pulmonary disease, idiopathic interstitial pneumonia, urinary tract infection, prostatic hypertrophy, decubital ulcer, lung cancer, gastric cancer, colon cancer, hematopoietic malignancy, cataract, and glaucoma, as well as clinical dementia ratings and histories of smoking and alcohol consumption) and pathological findings (720 items frequently encountered in autopsy examinations of elderly subjects). Details on the GEAD have been reported elsewhere.

5All CODs recorded on death certificates based on clinical and autopsy records were first evaluated by M.S., a pathologist and co-author of this study, for reporting consistency and adherence to instructions for proper completion of the death certificate. The CODs were subsequently evaluated by T.A., also a pathologist and co-author of this study, to confirm the accuracy of the findings and were entered into the database using the International Classification of Diseases, Tenth Revision (ICD-10) codes. UCODs based on death certificates were defined as the diagnoses listed last in Part I of death certificates according to guidelines published by the Ministry of Health, Labour and Welfare in Japan. UCODs based on postmortem examination in conjunction with clinical information were diagnosed by the same two pathologists, M.S. and T.A., as the reference standard. UCODs specified for each subject were coded using Simcode as well as ICD-10. Simcode is the classification code developed by the Japanese Ministry of Health, Labour and Welfare to define vital statistics. The overall agreement between UCOD identified on death certificates and the reference standard was classified into the following categories: 1. Perfect ICD-10 code agreement; 2. Disagreement involving the same organ system; 3. Disagreement, but listed as a COD on death certificate; and 4. Complete disagreement. We defined these agreement proportions as the concordance rates, sensitivity as the proportion of the cases positively identified using both methods (UCOD identified on death certificate [+] and UCOD identified using the reference standard [+]) to the cases positively identified using the reference standard, and specificity as the proportion of the cases negatively identified using both methods (UCOD identified on death certificate [−] and UCOD identified using the reference standard [−]) to the cases negatively identified using the reference standard.

6McNemar’s test was used to evaluate differences between UCOD proportions estimated based on data solely from the death certificates and those estimated based on reference standard data. We also calculated the 95% Wald confidence intervals (CIs) with Bonett-Price Laplace adjustment for differences between proportions. Multivariate unconditional logistic regression analyses assessed the effect of age at death (<80 vs 80–89 and ≥90 years), sex, comorbidity, and major UCODs identified on death certificates (cancer, heart disease, pneumonia, and others) on UCOD misclassification. Comorbidity was defined as the number of clinical findings present among the 26 findings registered in the GEAD. In the logistic regression model, we had classified the number of comorbidity into three groups: no or low comorbidity (0–1 finding), moderate comorbidity (2–4 findings), and high comorbidity (≥5 findings).

7Sensitivity and specificity with 95% Clopper-Pearson exact CIs were calculated for UCODs estimated to be present in at least 5% of the study population. We used SAS and JMP software for Windows (versions 9.3 and 10, respectively; SAS Institute, Cary, NC, USA) for all statistical analyses. Statistical significance was set at P < 0.05.

8The Japanese Postmortem Examination and Corpse Preservation Act generally permits use of autopsy materials for medical education and research. This study was approved by the ethics committee of Tokyo Metropolitan Geriatric Hospital (#240423).

9Table 1 shows subject characteristics. The average age at death was 79.8 years (range, 46–100 years; median, 80 years). Median number of major clinical findings was 3 (range, 0–8).

10UCOD distributions by sex are shown in Table 2. Simcodes generally conformed to ICD-10 codes, which are also shown in Table 2. The results indicate that cancer mortality would be underestimated (the absolute difference between death certificate information and the reference standard was 5.3% in women [95% CI, 0.49–10.0%; P = 0.025] and 6.1% in men [95% CI, 2.2–9.9%; P = 0.0017]), whereas the mortality for respiratory system diseases, especially pneumonia, would be overestimated (the absolute difference between death certificate information and the reference standard was 6.4% [95% CI, 1.6–11.1%; P = 0.0073] in women and 8.7% [95% CI, 4.1–13.3%; P = 0.0002] in men).

11Of 450 UCODs identified on death certificates, 214 (47.6%) agreed completely with UCODs identified based on clinical and post-autopsy reports at ICD-10 three-digit code levels. When we applied Simcode (broader categories than the ICD-10 code categories shown in Table 2) to UCODs, the concordance rate increased to 59.3% and was further improved to 69.6% when major Simcodes (largest CODs category, indicated by boldface in Table 2, used for rough national mortality statistics) were used (Figure). Of 236 instances of UCOD disagreement, 83 (35.2%) cases were assigned to the same organ system, 38 (16.1%) were assigned as CODs but not UCODs on the death certificates, and 115 (48.7%) disagreed completely.

12We also explored how concordance rates varied depending on UCODs. The concordance rate for cancer was 80.8% at the ICD-10 code level and increased to 93.6% at the major Simcode level. The concordance rate at the ICD-10 code level for heart disease was not high (54.7%); however, it improved to 83.0% at the major Simcode level. Among major UCODs, pneumonia, which is the third leading COD in Japan in 2012, had the lowest concordance rate (8.8% at the ICD-10 code level) (Figure).

13We next examined the effects of sex, age, comorbidity, and UCODs on misclassification of UCODs identified on death certificates (Table 3). We found that sex, comorbidity, and age did not affect the UCOD misclassification rate (P = 0.53, P = 0.75, and P = 0.13, respectively), although the misclassification rate showed an increasing trend, especially for cases >90 years old (adjusted odds ratio [vs <80 years old] 1.44; 95% CI, 0.72–2.88). The strongest factor for misclassification was UCODs (P < 0.0001); the results also show that cancer and heart disease were less often misclassified than other minor UCODs (adjusted odds ratio 0.10; 95% CI, 0.06–0.16 and adjusted odds ratio 0.34; 95% CI, 0.18–0.65, respectively), whereas pneumonia was significantly misclassified compared to other minor UCODs (adjusted odds ratio 4.44; 95% CI, 1.66–11.8) (Table 3). On exploring the factors influencing accuracy of sensitivity and specificity for each disease, we found that age (>90 years) had a profound influence on specificity for pneumonia (odds ratio 3.23; 95% CI, 1.50–6.69; P = 0.0016), although the sample size was relatively small for such disease-specific analyses.

14Finally, we evaluated the sensitivity and specificity of UCODs estimated to be present in at least 5% of the population (Table 4). Statistics were calculated for each UCOD identified on death certificates compared with the reference standard of assessment by two pathologists based on autopsy data and past clinical records. Overall, specificity for each UCOD was at least 85%. Sensitivity for any cancer was high (80%), although values varied according to organ. Sensitivity for heart disease was 60%, and sensitivity for pneumonia was very low (46%). Results also suggested that diseases of the digestive system were difficult to specify as UCOD (sensitivity, 51.9%). Among 13 deaths attributable to digestive diseases, 5 (38%) were reported as deaths due to unknown causes, 3 (23%) as deaths due to infectious diseases, and 3 (23%) as deaths due to heart disease.

15Table 5 also shows that deaths due to cancer and heart disease were underestimated regardless of true UCODs, and 18 (38%) of 47 deaths due to pneumonia and 28 (55%) of 51 deaths due to respiratory diseases would be considered deaths due to cancer or heart disease.

16We evaluated the accuracy of UCODs, particularly major UCODs, recorded on the death certificates of elderly patients in Japan. To our knowledge, this is the first report to quantitatively estimate accuracy for several UCODs specified on death certificates. Data from death certificates are used for many clinical and population-based studies and national vital statistics, although the difficulties in properly completing the COD section of the death certificate to ensure accuracy of COD data have been well documented.– Several recently proposed statistical methods to account for outcome variable misclassification enable bias correction of effect estimates due to misclassified outcomes, such as those measured by death certificates.– However, it is difficult to quantitatively evaluate the accuracy of data from death certificates, as we have done here, because reference standard data is not easily obtainable, especially in studies that utilize large national databases. Our results might be informative either for applying bias correction methods or sensitivity analyses to assess effect estimate bias in studies using data from death certificates.

17According to national vital statistics’ reports, the four leading UCODs in Japan in 2000 were malignant lymphoma (29.6% of deaths among 80- to 84-year-olds), heart disease (16.1% of deaths among 80- to 84-year-olds), cerebrovascular disease (11.6% of deaths among 80- to 84-year-olds), and pneumonia (11.4% of deaths among 80- to 84-year-olds). In our study, the top four UCODs were malignant lymphoma (28.5% among individuals in their 80s), pneumonia (16.5% among individuals in their 80s), heart disease (13.3% among individuals in their 80s), and digestive system disease (7.6% among individuals in their 80s). Thus, except for death due to cerebrovascular disease, the distribution of UCODs in our population was similar. This is because the Tokyo Metropolitan Geriatric Hospital is not an acute care hospital, and most cases had chronic diseases. The population analyzed here is not representative of the whole population of elderly people in Japan, and we could not assess the accuracy of UCODs for acute diseases in this study. However, our data showed that deaths due to cancer and heart disease based solely on death certificate records would be underestimated, a finding that has also been reported in previous studies., Hu et al assessed the reliability of COD for the Surveillance, Epidemiology, and End Results (SEER) database using a relative survival approach and showed that the number of cancer-specific deaths documented in SEER was over-coded for early stage cancers or cancers with favorable prognoses, whereas SEER tended to undercode the number of cancer-specific deaths for cancers with generally poor prognosis or advanced-stage cancers. In our study data, most cancer-specific deaths were of poor prognosis or advanced-stage cancer, so our observation is consistent with previous research.

18In general, COD in elderly patients is subject to speculation because of the competing effects of comorbidity-associated mortality. However, while our data showed neither significant comorbidity nor age effects, we observed that the misclassification rate in very old patients tended to be higher than in younger patients even after adjusting for UCOD and comorbidity. This suggests that “more likely” CODs without detailed investigation are recorded on death certificates regardless of patient history, particularly if the patient was more than 90 years of age and died of old age.

19To our knowledge, there have been no previous reports on the accuracy of COD data from death certificates for pneumonia, despite being a leading COD in many countries. As discussed above, the UCOD recorded for elderly patients could be the “more likely” COD, and pneumonia would be a most likely UCOD in very elderly patients because many of them are likely to die of pneumonia. Another reason for the high pneumonia misclassification rate was that many cases of aspiration pneumonia were reported as deaths due to pneumonia. In contrast to the misclassified cases of death due to digestive or other minor diseases, misclassified death due to pneumonia is likely to be caused by misjudgment and not by errors in diagnostic techniques. Myers et al showed that the accuracy of death certificates could be improved by implementation of a simple educational intervention. In Japan, many medical doctors previously reported heart failure as the UCOD on death certificates regardless of the true UCOD., However, this poor practice has improved in the past several decades by adding a note on death certificates according to a revised ICD-10 code, which states, “Do not enter the mode of dying, such as cardiac or respiratory arrest, shock, or heart failure.” Therefore, the pneumonia misclassification rate could be reduced by education or by including notes or instructions in the guidelines for completing death certificates when pneumonia appears as a condition on the death certificate.

20Although having multiple-cause autopsy mortality data was a strength of this study, the potential for autopsy bias limits our ability to generalize the results to the rest of the population. As mentioned above, we could not assess the accuracy of UCODs for acute diseases, such as cerebrovascular death. Additionally, we were unable to measure the accuracy of UCOD for minor diseases and diseases for which only clinical diagnoses were available, such as diabetes or some psychiatric diseases. To assess the validity of death certificate data for such diseases, additional disease-specific studies modeled on previous reports are necessary.,, The data we investigated were collected more than 10 years ago. If the medical record training for doctors had been well-established during the period, we might have obtained more accurate sensitivities and specificities. However, to our knowledge, the situation has not changed much, so improvements in medical recordkeeping may have little effect on the interpretation of our results.

21Researchers should be aware of the accuracy of COD data on death certificates used as research resources, particularly for elderly research subjects who died from diseases other than cancer (especially pneumonia).