Incidence, co-occurrence, and evolution of long-COVID features: A 6-month retrospective cohort study of 273,618 survivors of COVID-19

1There is increasing concern, and emerging evidence, that some people who contract Coronavirus Disease 2019 (COVID-19) do not make a rapid or full recovery—so-called “long-COVID” or “long-haulers”. The term, and the related “post-acute COVID-19 syndrome” encompasses a range of features indicative of involvement of many organs affecting people weeks and months after infection. Although a long-COVID syndrome remains to be defined, prominent reported features include breathlessness, headache, chest pain, abdominal symptoms, myalgia, fatigue, cognitive difficulties, as well as anxiety and depression. We use the term “features,” since as currently conceptualized, they are a mixture of symptoms and diagnoses.

2Although several studies have addressed the issue of long-term sequelae of COVID-19, they all have some important limitations. For example, the results of a telephone survey in France (with a 57% response rate reaching 478 patients) showed that at 4 months after hospitalization for COVID-19, about half the patients had at least one feature of long-COVID. In an app-based cohort study with 4,182 cases of COVID-19, 13% of respondents self-reported long-COVID features, with some evidence for higher rates in women and older people. Another investigation followed 1,733 patients hospitalized for COVID-19 for 6 months and found fatigue or muscle weakness in 63%, sleep difficulties in 26%, anxiety or depression in 23%, and lower rates of myalgia and headache. These studies lack a control group and have limited generalizability, focusing either on hospitalized patients or individuals who voluntarily responded to a telephone survey or used an app.

3Some investigators have compared long-term sequelae of COVID-19 to those occurring after influenza. A 6-month retrospective cohort study based on electronic health records (EHRs) of 236,000 COVID-19 patients found higher rates of anxiety and mood disorders, insomnia, and dementia post-COVID-19 than after influenza. Another report, based on EHRs from American veterans (88% male) has also identified increased rates of sequelae in multiple body systems after COVID-19 compared to influenza. To our knowledge, though there have been no robust estimates, in a general population, of the incidence and co-occurrence of long-COVID features, their relationship to age, sex, or severity of infection, and the relative risk after COVID-19 compared to influenza. Using EHRs, we sought to address these questions.

4The study protocol (not included in the supporting information), including cohorts definition, covariates, and outcomes, was planned before it was executed and almost exactly followed the protocol used in previous studies based on the same dataset (except for the outcomes that are specific to this study). The differences with the protocols of the previous studies were defined before the analysis was executed and were motivated by accrued knowledge about COVID-19 and its risk factors (e.g., psychiatric illness was added as a covariate given its association with risk of COVID-19). No change to the protocol for the primary analysis and the analysis of subgroups was brought after the analysis started. The other secondary analyses were motivated by the results of the primary analysis and by interactions with the reviewers of this article.

5The study used TriNetX Analytics, a federated network of linked EHRs recording anonymized data from 59 healthcare organizations (HCOs), primarily in the United States of America, totaling 81 million patients. Available data include demographics, diagnoses (using ICD-10 codes), and measurements (e.g., leukocytes count). The HCOs in the network are a mixture of hospitals, primary care, and specialist providers, and they contribute data from uninsured and insured patients. The process by which the data is deidentified is attested to through a formal determination by a qualified expert as defined in Section §164.514(b)(1) of the HIPAA Privacy Rule. This formal determination by a qualified expert, refreshed in December 2020, supersedes TriNetX’s waiver from the Western Institutional Review Board (IRB). For further details about TriNetX, its data, provenance, and functionalities, see Supporting Methods A in S1 Text.

6The primary cohort was defined as all patients who had a confirmed diagnosis of COVID-19 (ICD-10 code U07.1). It was compared to a matched cohort of patients diagnosed with influenza (ICD-10 codes J09-J11) who did not have a diagnosis of COVID-19 or a positive test for COVID-19. Cohorts included all patients over the age of 10 who had the index event (COVID-19 or influenza) on or after January 20, 2020 (the date of the first recorded COVID-19 case in the USA) and who were still alive at the end of follow-up (December 16, 2020). We note that patients in the influenza cohort must have presented in order to have been diagnosed, and, hence, the cohort is likely enriched for those who had a more severe form of the illness. Further details on cohorts are provided in Supporting Methods B in S1 Text.

7A set of established and suspected risk factors for COVID-19 and for more severe COVID-19 illness was used: age, sex, race, ethnicity, obesity, hypertension, diabetes, chronic kidney disease, asthma, chronic lower respiratory diseases, nicotine dependence, substance misuse, previous psychiatric illness, ischemic heart disease, and other forms of heart disease, socioeconomic deprivation, cancer (and hematological cancer in particular), chronic liver disease, stroke, dementia, organ transplant, rheumatoid arthritis, lupus, psoriasis, and disorders involving an immune mechanism. To capture these risk factors in patients’ health records, 58 variables were used. More details including ICD-10 codes are presented in Supporting Methods C in S1 Text. Cohorts were matched for all these variables, as described below.

8In the absence of consensus as to what constitutes long-COVID, we adopted a pragmatic approach that attempted to capture features common to most descriptions, and which accounted for the fact that some features are diagnoses whereas others are symptoms. The 9 clinical features were as follows:

9chest/throat pain;

10abnormal breathing;

11abdominal symptoms;

12fatigue/malaise;

13anxiety/depression;

14pain;

15headache;

16cognitive dysfunction; and

17myalgia.

18We defined each clinical feature using all ICD-10 codes we considered to be encompassed by the term. For example, to capture “abdominal symptoms,” we included abdominal pain, change of bowel habit, and diarrhea. We included atopic dermatitis as a negative control. See Supporting Methods D in S1 Text for ICD-10 codes used for each feature.

19The main outcomes for the analysis were the 9 long-COVID features. Their incidence and co-occurrence were analyzed both over the whole period from 1 to 180 days after the index event and specifically over the “long” phase, i.e., 90 to 180 days post-diagnosis (see below). In Kaplan–Meier analysis, outcomes are only counted the first time they occur in the follow-up period. Thus, if a patient has a feature recorded once in the acute phase of the illness and again a few months later, the latter occurrence would not count toward the incidence at later stages of follow-up. This implies that, in the 1- to 180-day follow-up, ongoing incidence after 3 months represents emergence of new long-COVID features in an individual with no record of these features before. Results from the “long” phase, on the other hand, assess the overall incidence of long-COVID features in the 90 to 180 days post-diagnosis (i.e., including patients who already experienced these features in the first 3 months and are experiencing them again). Jointly analyzing the incidence in these 2 time windows allowed us to separately estimate the rate of first occurrences and the rate of recurrences of long-COVID features (where recurrences are defined as a feature, which is recorded in the 90- to 180-day window but which was also recorded in the 1- to 90-day window).

20The choice to start the “long” phase at 90 days was motivated by an emerging consensus that this corresponds to the longer phase of the illness. The choice to end the follow-up at 6 months was a pragmatic one, aiming to have a long-enough follow-up while making sure that enough individuals contribute data at the end of the follow-up (taking a much longer follow-up would mean that only patients diagnosed early in the pandemic would be contributing data). This is also consistent with our prior study on the long-term sequelae of COVID-19 wherein a 6-month follow-up was used.

21Propensity score 1:1 matching (with greedy nearest neighbor matching, and a caliper distance of 0.1 pooled standard deviations of the logit of the propensity score) was used to create cohorts with matched baseline characteristics and carried out within the TriNetX network. Characteristics with a standardized mean difference (SMD) between cohorts ≤0.1 was considered well matched. Because we used EHR with coded health events, if an event was not present, it was considered absent. Missing data for race and ethnicity were assigned their own category and that category was included in the propensity score matching, so that the 2 matched cohorts had approximately equal numbers of patients with unknown race/ethnicity.

22We first investigated the incidence and co-occurrence of long-COVID features over the 180 days following a COVID-19 diagnosis and specifically over the 90- to 180-day period using Kaplan–Meier analysis. Co-occurrence consists in the occurrence in the same patient of a pair of features. We also separately calculated the incidence of each feature occurring in the 1- to 90-day period only, in the 90- to 180-day period only, and the rate of recurrences (i.e., the incidence of features, which occurred in the 90- to 180-day period and which had already occurred in the 1- to 90-day period). This was achieved using simple algebraic relations between these and the recorded incidences over the whole 1 to 180 days and the “long” 90- to 180-day period (see Supporting Methods E in S1 Text for details).

23Incidences and co-occurrences were compared with those occurring in the matched control cohort using the Cox proportional hazard model. The proportional hazard assumption was tested using the generalized Schoenfeld approach. When violated, time-varying hazard ratios (HRs) were assessed using natural cubic splines fitted to the log-cumulative hazard. The absolute risk increase was also calculated between the 2 matched cohorts for each clinical feature by subtracting the cumulative incidence in the influenza cohort from that in the COVID-19 cohort at the end of the time windows (for both the 1- to 180-day and the 90- to 180-day windows).

24To assess whether long-COVID features co-occur more often after COVID-19 than after influenza, above and beyond what their individual incidences would predict, we calculated Dice’s coefficient (D), which is the probability of co-occurrence divided by the mean of individual incidences. It varies between 0 and 1. For a pair of features, D = 1 means that one occurs if and only if the other one occurs, and D = 0 means that the two never co-occur.

25Clinical features can be represented as the nodes of a network whose connections are weighted by Dice’s coefficient for the pair of features they connect. A strongly interconnected network indicates that when a clinical feature occurs, it tends to co-occur with other features, whereas a weakly interconnected network indicates that the features tend to occur in isolation. The average degree of a network (the sum of the connections of each node with other nodes, averaged over all nodes) is a standard measure of how interconnected the network is. We calculated the average degree of the clinical feature network for both the COVID-19 and influenza cohorts, calculated the effect size of the difference, and tested whether there was a significant difference between the two using permutation tests with 1,000 permutations.

26All confidence intervals (CIs) were calculated using nonparametric bootstrap with 1,000 replicates. Statistical analyses were conducted in R version 3.6.3. Statistical significance was set at two-sided p-values < 0.05. Details of the statistical analyses can be found in Supporting Methods E in S1 Text.

27We assessed whether the incidence and co-occurrence of long-COVID features vary with differences in demographics or with the severity of the COVID-19 illness by conducting 8 further cohort studies, comparing subgroups of COVID-19 patients: (1) female versus male; (2) non-white versus white; (3) age 45 and over versus age 10 to 44; (4) age 65 and over versus age 45 to 64; (5) age 22 to 44 versus age 10 to 21; (6) patients requiring versus not requiring hospitalization; (7) patients requiring versus not requiring intensive treatment unit (ITU) care; and (8) patients with versus without leukocytosis (defined as a leukocyte count over 11,000 per microliter). Subgroups (6) to (8) were measured between 4 days before and 14 days after the date of diagnosis of COVID-19 or influenza. Leukocytosis was used as leukocyte count was one of the most commonly reported inflammatory markers in the sample. Each pair of cohorts were matched for the 58 covariates and compared using the same approach as described for the primary analysis.

28As an approach to assess differences in the burden of each clinical feature among patients who have them, we counted the number of occurrences of each clinical feature in the 6 months after the index event (this part of the analysis was conducted on July 18, 2021, so that all patients diagnosed on or before December 16, 2020 had over 6 months of follow-up available) and compared this between the COVID-19 and influenza cohorts. Restricting the cohorts to those with at least one record of the feature guarantees that we are not merely replicating the main analysis in which the cohorts are compared based on the presence of the feature. We view the number of occurrences as being a composite of persistence of the original diagnosis, and a true recurrence. The mean and bootstrap-derived 95% CIs for the count of occurrences are reported, and the null hypothesis that there is no difference in the number of occurrences between the 2 cohorts was tested using a Poisson regression.

29Details of the secondary analyses are presented in Supporting Methods F in S1 Text. This report complies with the STROBE reporting guideline (see S1 STROBE Checklist).

30A total of 273,618 patients with COVID-19 were identified, and 114,449 patients with influenza were available for matching. Table 1 (and Tables A and B in S1 Tables) summarizes the baseline characteristics, and incidence of the 9 long-COVID features, for the full COVID-19 cohort (n = 273,618, mean [SD] age 46.3 [19.8] years, 55.6% female), as well as for the matched COVID-19 and influenza cohorts (n = 106,578). Adequate propensity score matching (SMD ≤ 0.1) was achieved for all comparisons.

31In the 6 months after COVID-19 diagnosis, 57.00% (95% CI 56.59 to 57.43) had at least one feature of long-COVID recorded. This value includes the incidence of features recorded in the acute as well as the later phase of the illness. In the 90- to 180-day “long” phase post-diagnosis, 36.55% had a long-COVID feature recorded (95% CI 36.18% to 36.94%). The incidence of individual features ranged from 3.24% (95% CI 3.09 to 3.38; including 1.54% in the “long” phase, 95% CI 1.44 to 1.64) for myalgia, to 22.8% (95% CI 22.48 to 23.14; including 15.49% in the “long” phase, 95% CI 15.21 to 15.77) for anxiety/depression.

32In the “long” phase, the rates of first occurrence versus recurrences varied between clinical features (Fig 1). Overall, of patients with long-COVID features recorded between 90 and 180 days, 39.9% had not had any feature recorded in the first 90 days; the remaining 60.1% had at least one long-COVID feature in the first 90 days and developed additional or recurrent features in the next 90 days. The incidence of all features, except pain, was lower in the 90- to 180-day period than in the 1- to 90-day period (Table C in S1 Tables).

33As shown in Fig 2 and Fig A in S1 Fig, the incidence of each and any long-COVID feature was significantly higher after COVID-19 than after influenza (overall HR = 1.65 [95% CI 1.62 to 1.67]; individual HRs between 1.44 and 2.04; all p < 0.001 for the whole 1- to 180-day period; and overall HR = 1.56; individual HRs between 1.36 and 1.97; all p < 0.001 for the “long” phase). Long-COVID features are thus more common after COVID-19, although 42.8% of patients with influenza also had one of these features recorded over the same period (including 29.7% during the “long” phase). In contrast, the HR for atopic dermatitis (negative control) was not statistically significantly different from 1 (HR 1.05, 95% CI 0.91 to 1.23, p = 0.51 for the 1- to 180-day period; and HR 1.07, 95% CI 0.87 to 1.32, p = 0.52 for the “long” phase). The absolute risk increase of any long-COVID feature after COVID-19 compared to influenza was 16.60% (95% CI: 15.84% to 17.33%) for the whole follow-up window and 12.64% (11.73% to 13.57%) for the “long” phase and varied between 1.43% and 8.71% (0.78% to 4.97% in the “long” phase) for different features (Table D in S1 Tables).

34Fig 3A and 3B show how often pairs of long-COVID features co-occur. For example, 5.99% of patients had both abnormal breathing and chest/throat pain (including 2.04% during the “long” phase). Fig 3C and 3D give the HRs for each co-occurrence compared to influenza. Long-COVID features are seen to co-occur more commonly after COVID-19, as shown by HRs significantly greater than 1, ranging from 1.58 to 2.80 (and from 1.30 to 2.39 for the “long” phase). CIs and p-values for all estimates are presented in Tables E-H in S1 Tables.

35Most hazards for both incidence and co-occurrence of long-COVID features were found to be proportional, and, when they were not, they remained larger than 1 at 6 months for most outcomes (see Table I in S1 Tables and Figs B and C in S1 Fig).

36The clinical feature network was found to be more interconnected post-COVID-19 than following influenza (mean [95% CI] degree: 1.70 [1.54 to 1.87] versus 1.39 [1.26 to 1.53], p < 0.001, Fig D in S1 Fig). The network became significantly more interconnected over time (Fig 4 and Fig E in S1 Fig; biweekly change in average degree: 0.13, 95% CI 0.075 to 0.20, p < 0.001). During the 90- to 180-day window, the clinical feature network in the COVID-19 cohort was similarly interconnected as in the influenza cohort (see Fig F in S1 Fig and Table J in S1 Tables; p = 0.21).

37We next explored how sex, age, race, and indices of COVID-19 severity impacted on long-COVID features (Fig 5 and G-AE in S1 Fig and Tables K-T in S1 Tables). The incidence of “any” long-COVID feature varied from 46.42% in the 10- to 21-year age group, to 61.05% in the over 65s, 63.64% of those hospitalized, and 73.22% of those admitted to ITU. Females were significantly more likely to have headaches, abdominal symptoms, and anxiety/depression, whereas males were significantly more likely to have breathing difficulties and cognitive symptoms. Younger patients were significantly more likely to have headaches, abdominal symptoms, and anxiety/depression, whereas older patients were more likely to have breathing difficulties, cognitive symptoms, pain, and fatigue. Only minor differences were observed between white and non-white patients.

38Differences in severity of the COVID-19 illness were also associated with differences in incidence of long-COVID features. Patients with more severe illnesses (as proxied by hospitalization, ITU admission, or leukocytosis) had significantly more features overall, and significantly more cognitive difficulties, but were less likely to have myalgia or headaches. There were no significant differences in the clinical feature networks in any of these subgroup comparisons (Table J in S1 Tables).

39Finally, among people who had a given long-COVID feature reported at least once, the total number of recorded occurrences of that feature was significantly larger among patients with COVID-19 than patients with influenza (Table U in S1 Tables): p < 0.001 for all clinical features except chest/throat pain (p = 0.043), pain (p = 0.26), and myalgia (p = 0.77).

40COVID-19 appears to be associated with long-term effects that are common and diverse, with 57% of patients having at least one long-COVID feature recorded in the 180 days after infection (Table 1 and Fig 1) and 37% having them in the 90 to 180 days after diagnosis, of whom 40% had not had one in the first 3 months. These features are all more common after COVID-19 than after influenza. The fact that most hazards were proportional between the 2 cohorts or that the HRs remained above 1 after 180 days indicates that the risk of long-COVID features occurring, or co-occurring, continues to increase 180 days after the illness and that, for most of them, they follow the same trend as in the acute phase of the illness.

41Cognitive symptoms (for definition, see Supporting Methods D in S1 Text) were observed in 7.88% of patients after COVID-19, with markedly higher incidences in the elderly and in those hospitalized or needing ITU admission (Figs V and Y in S1 Fig). “Other symptoms and signs involving cognitive functions and awareness” (ICD-10 code R41) account for 67% of post-COVID cognitive symptoms in the matched COVID-19 cohort, which likely reflects the “brain fog,” word finding difficulties or poor concentration reported in descriptions and surveys of long-COVID. The present findings complement evidence of impaired cognitive performance in a large population of people who self-reported a history of COVID-19. In total, the data highlight cognition as an important issue after COVID-19, which requires surveillance and investigation.

42The long-COVID features involving pain were notable for 3 reasons. First, the overall incidence of pain (of any kind) recorded after COVID-19 was 34.2% (Table V in S1 Tables), higher than any of the other features, and higher than after influenza (24.0%). Second, pain was the only feature that had a higher incidence in the 3- to 6-month period than in the 0- to 3-month period (Table C in S1 Tables). Pain, therefore, appears to be a prominent and relatively persistent element of long-COVID. Third, headache and myalgia had characteristics that differed from the other pain categories: They were more common in women and in younger patients, and notably so in those who had been less acutely ill (as proxied by not requiring hospitalization or ITU admission, and without leukocytosis; Fig 4 and Fig AE in S1 Fig). In each case, this was in the opposite direction to the overall burden of long-COVID features. As such, post-COVID headache and myalgia may result from a different mechanism than the other long-COVID features.

43Long-COVID features were observed in all COVID-19 subgroups examined, but the incidence of features differed in relation to demographic and illness severity factors. Overall, there was a higher incidence of long-COVID features in the elderly, in more severely affected patients, and in women. However, it is notable that long-COVID features were also recorded in children and young adults, and in more than half of nonhospitalized patients, confirming that they occur even in young people and those who had a relatively mild illness (Tables K-L in S1 Tables). This is significant in public health terms, given that most people with COVID-19 are in the latter group. It is also of interest that almost 40% of patients with long-COVID symptoms recorded between 3 and 6 months had not had any such diagnosis in the first 3 months. Some of this may reflect a delay in presentation but also suggests that some patients may have a delayed onset of long-COVID features.

44Besides high incidences of individual features, long-COVID features co-occurred (Fig 3) and formed a network that emerged over time (Fig 4). The evolution of this network indicates that, compared to the acute phase of the illness, when a long-COVID feature was observed in the later stage of the follow-up, it was more likely to co-occur with other features. This is consistent with the existence of a long-COVID “syndrome,” although this cannot be established by a study of this kind. Similarly, our data cannot identify whether there are subtypes of long-COVID. However, Fig 4 shows that abnormal breathing, chest/throat pain, fatigue, and anxiety/depression are particularly interconnected and form early, whereas myalgia and cognitive symptoms remain only weakly connected. It is therefore possible that the latter features may have a different origin and/or mechanism relative to the other long-COVID features.

45The finding that 57% of patients have a long-COVID feature recorded in the 180 days after a diagnosis of COVID-19 should not be misinterpreted as a 57% incidence of long-COVID since this time window includes the acute phase of the illness. However, jointly analyzing the 2 time windows (1 to 180 days and 90 to 180 days) gives insight into the pattern of occurrences and recurrences of long-COVID features throughout the first 180 days post-COVID-19. Specifically, some features (e.g., abnormal breathing) are more common in the first 90 days post-COVID than in the next 90 days, whereas others (e.g., anxiety/depression) are more common in the “long” phase (Fig 1). It can also be seen that fatigue, when it occurs in the “long” phase, tends to be a first occurrence rather than a recurrence, whereas the opposite is true of depression/anxiety. Combined with the finding of a higher number of occurrences of most long-COVID features post-COVID than post-influenza, these results suggest that for some patients with long-COVID, the course of the illness is marked by relapses, in line with results from a recent study based on survey data. However plausible, this conclusion cannot be drawn with confidence from our data since a recurrent record of a long-COVID feature might represent a persistent symptom rather than a relapse. In either case, they show that the burden of most long-COVID features appears greater for patients after COVID-19 than after influenza.

46We note several limitations, beyond those inherent to research using EHRs (summarized in Supporting Methods A in S1 Text). First, there is an increasingly long list of putative long-COVID features, and we captured only a proportion of them, albeit the majority of those that appear to be commonly described. For example, we did not measure anosmia or hair loss. Second, many patients may have unrecorded long-COVID features, making our incidence figures lower bound estimates. Third, we do not know with any clarity the persistence nor severity of the long-COVID features. Fourth, using an EHR network with both insured and uninsured patients represented provides a higher degree of external generalizability than many previous studies. However, there are limits; many people are likely to have had COVID-19 without it being recorded in their EHR, either because it went undiagnosed or because they were tested at other facilities. In addition, patients who died in the 6 months following their illness were not represented in the data. Fifth, because fewer patients had influenza than COVID-19 during the study period, the 1:1 matching resulted in a matched COVID-19 cohort, which had baseline characteristics similar to the influenza cohort, rather than the other way around. This implies that, strictly speaking, all results comparing the 2 cohorts apply to a population that has the baseline characteristics of the matched COVID-19 cohort, not the whole COVID-19 cohort and the two differ slightly (e.g., the matched cohort is younger than the whole cohort). A final limitation is that, after COVID-19, patients might be more likely to report symptoms, or health professionals more likely to make diagnoses, compared to patients recovering from influenza.

47The last limitation deserves further consideration for it might artificially increase the HR between cohorts. But 2 factors suggest that any effect is likely to be limited. First, the 2 cohorts were diagnosed with atopic dermatitis (a negative control) at the same rate over the same time period. Second, at the time our data collection ended (December 2020), long-COVID was far less in the public consciousness than it is now (see Fig AF in S1 Fig) and thus less likely to have been a major factor influencing presentations with symptoms.

48The data presented here shed light on the incidences and relative risks of long-COVID features, but say nothing about the causation or mechanisms involved, nor about the predictors beyond the limited demographic and severity markers we measured. Research aimed at these issues is required. For example, to what extent do preexisting health conditions or characteristics, or specific features of the acute infection, impact on long-COVID? Finally, we note that almost 43% of patients after influenza had at least one long-COVID feature recorded (Table 1) including 29.7% during the 90- to 180-day period. In this regard, we suggest researchers take a broad and balanced view as to the nature and specificity of long-COVID. In the meantime, clinical services need to be prepared and resourced to enable management of long-COVID features according to the best available evidence as it emerges.

49The views expressed are those of the authors and not necessarily those of the UK National Health Service, NIHR, or the UK Department of Health.