Development and Validation of the Elecsys Anti-SARS-CoV-2 Immunoassay as a Highly Specific Tool for Determining Past Exposure to SARS-CoV-2

1In December 2019, reports emerged of patients presenting with pneumonia of unknown etiology in Wuhan, China. This disease was subsequently shown to be caused by a novel coronavirus, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), and was designated coronavirus disease 2019 (COVID-19). Since then, the COVID-19 outbreak has rapidly developed into a pandemic, which has infected millions of people and posed critical challenges for governments and health care systems around the world.

2SARS-CoV-2 is an enveloped, single-stranded RNA virus of the Coronaviridae family. All coronaviruses share similarities in the organization and expression of their genomes, which encode 16 nonstructural proteins and 4 structural proteins: the spike, envelope, membrane, and nucleocapsid antigens. Evidence to date suggests that SARS-CoV-2 is transmitted between people primarily through respiratory droplets and contact routes, although indirect transmission via contaminated surfaces is also possible. Infected individuals may exhibit a variety of symptoms, including fever, cough, and breathlessness, and disease severity can range from asymptomatic/mild cases to severe disease and death.

3There is an urgent unmet clinical need to more effectively determine SARS-CoV-2 seroprevalence in the general population in order to improve our understanding of virus circulation dynamics, gain a more accurate estimate of the mortality rate from COVID-19, and identify individuals at risk of infection. Serological assays for SARS-CoV-2 have been suggested as a potential tool to help identify the extent of virus exposure in a given population and thereby indirectly provide information on the appropriate application, enforcement, or relaxation of containment measures. Serological assays may also help elucidate a potential correlate for immunity following infection.

4Recent evidence suggests that most SARS-CoV-2 convalescent individuals have detectable neutralizing antibodies (nAbs) for the virus. Due to affinity maturation, the binding strength of antibodies increases over time following infection or vaccination. High-affinity nAbs are critical for the control of infection, since they can recognize and bind to specific viral epitopes, thereby “neutralizing” the virus and rendering it nonpathogenic. Previous studies involving commercially available anti-SARS-CoV-2 immunoassays have found a positive correlation between antibody titration results from pseudoneutralization assays and SARS-CoV-2 nAbs; however, further investigation is warranted.

5The timing of seroconversion is crucial for determining optimum time points for sample collection for serological testing. Although the picture is rapidly developing and robust serology data are not yet available, the kinetics of antibodies to SARS-CoV-2 have begun to be described. Based on current evidence, immunoglobulin M (IgM) antibodies are detectable within 5 days after symptom onset and immunoglobulin G (IgG) antibodies within 5 to 7 days. There is a paucity of data on immunoglobulin A (IgA), but it appears to be observable approximately 3 to 6 days after symptom onset. Depending on the method applied, seroconversion is observed after a median of 10 to 13 days after symptom onset for IgM and 12 to 14 days for IgG; maximum seroconversion occurs at 2 to 3 weeks for IgM, 3 to 6 weeks for IgG, and 2 weeks for total antibodies. The levels and chronological order of IgM and IgG antibody appearance are highly variable, supporting the detection of both antibodies simultaneously.

6The Elecsys Anti-SARS-CoV-2 immunoassay (Roche Diagnostics International Ltd, Rotkreuz, Switzerland) was developed to provide an accurate and reliable method for the detection of antibodies to SARS-CoV-2, in order to facilitate population screening with high specificity and the identification of past infection status as a potential correlate for subsequent immunity. We aimed to evaluate the sensitivity, specificity, and cross-reactivity of the Elecsys Anti-SARS-CoV-2 immunoassay, in addition to agreement with results from a pseudoneutralization assay.

7The performance of the Elecsys Anti-SARS-CoV-2 immunoassay was prospectively evaluated at Roche Diagnostics (Penzberg, Germany). Sensitivity and specificity analyses were conducted using anonymized residual frozen samples from routine diagnostic testing or from blood donors, which were obtained from diagnostic laboratories in Germany (Labor Berlin–Charité Vivantes Services GmbH, Berlin, Germany, and KRH Labor GmbH, Hannover, Germany) and a blood product provider in the United States (Golden West Biologicals, CA, USA), respectively. All samples utilized for sensitivity analyses were from patients with PCR-confirmed SARS-CoV-2 infection. Cross-reactivity analyses were conducted using anonymized frozen samples containing potentially cross-reacting factors, which were purchased from various commercial vendors. For analysis of agreement between the Elecsys Anti-SARS-CoV-2 immunoassay and a vesicular stomatitis virus (VSV)-based pseudoneutralization assay, anonymized residual frozen samples from patients with PCR-confirmed SARS-CoV-2 infection were used.

8The study was conducted in accordance with applicable regulations, including relevant European Union directives and regulations, and the principles of the Declaration of Helsinki. All samples, excluding the specimens that were provided by commercial sample vendors, were transferred to Roche following anonymization. For studies with anonymized leftover specimens, no ethics committee vote is required. A statement was obtained from the Ethics Committee of the Landesärztekammer Bayern confirming that there are no objections to the transfer and coherent use of anonymized leftover samples.

9The Elecsys Anti-SARS-CoV-2 electrochemiluminescence immunoassay is intended for use on cobas e analyzers (Roche Diagnostics International Ltd, Rotkreuz, Switzerland) for in vitro qualitative detection of antibodies (including both IgA and IgG) to SARS-CoV-2 in human serum and plasma. The immunoassay utilizes a double-antigen sandwich test principle and a recombinant protein representing the nucleocapsid antigen for the determination of antibodies to SARS-CoV-2.

10For the present study, the Elecsys Anti-SARS-CoV-2 immunoassay was performed according to the manufacturer’s instructions, and assay results were interpreted as follows: cutoff index, <1.0 for samples that were nonreactive/negative for anti-SARS-CoV-2 antibodies; cutoff index, ≥1.0 for samples that were reactive/positive for anti-SARS-CoV-2 antibodies.

11The sensitivity of the Elecsys Anti-SARS-CoV-2 immunoassay was evaluated using residual samples from patients who had previously tested positive for SARS-CoV-2 infection by PCR. One or more consecutive samples were collected from patients at various time points after PCR confirmation. All samples taken from the same donor on the same day were excluded, leaving only multiple donations from one donor if taken on different calendar days. Samples derived from all patients with prior PCR-confirmed SARS-CoV-2 infection were used; no additional sample selection was made except for the availability of a sufficient amount of residual serum or plasma. Assay sensitivity at different time points after PCR confirmation was calculated as the percentage of samples that tested positive with the Elecsys Anti-SARS-CoV-2 immunoassay relative to the total number of PCR-confirmed positive samples.

12The specificity of the Elecsys Anti-SARS-CoV-2 immunoassay was evaluated using unselected residual samples from routine diagnostic testing or from blood donors. All samples were collected before December 2019 (i.e., before the first description of an infection with SARS-CoV-2) and were thus deemed negative for SARS-CoV-2-specific antibodies. Assay specificity was calculated as the percentage of true SARS-CoV-2 antibody-negative samples that tested negative with the Elecsys Anti-SARS-CoV-2 immunoassay.

13The cross-reactivity of the Elecsys Anti-SARS-CoV-2 immunoassay was evaluated using samples collected before December 2019 that had previously been characterized as positive for a wide range of different indications, including common cold (donors selected by symptoms only) and coronavirus (229E, NL63, OC43, and HKU1) panels, and samples from patients with autoimmune conditions and other diseases associated with a higher prevalence of autoantibodies and immune dysfunction, which may increase the risk of interference with serological testing. Assay specificity was calculated for each potential cross-reactive sample type and for the total cohort as the percentage of samples tested that were reactive with the Elecsys Anti-SARS-CoV-2 immunoassay.

14The diagnostic accuracy of the Elecsys Anti-SARS-CoV-2 immunoassay was compared with that of a VSV-based pseudoneutralization assay at an external laboratory in Geneva, Switzerland (Hôpitaux Universitaires Genève), using residual frozen samples from patients with PCR-confirmed SARS-CoV-2 infection. A titer of 1:20 (corresponding to the titer at which 50% pseudoneutralization activity was observed) was used as the positive cutoff for the pseudoneutralization assay. Discrepant samples were retested using an in-house SARS-CoV-2 whole spike protein-based immunofluorescence assay and were excluded if indeterminate results were obtained. Full details of the methodology used for VSV-based pseudoneutralization assay measurements have been published previously by Meyer et al.. The percent positive agreement (PPA), percent negative agreement (PNA), and percent overall agreement (POA) rates were determined, and 95% confidence intervals (CIs) were calculated.

15Sample size estimation for the specificity analyses was calculated using the binomial exact test. The following sample sizes for a seronegative cohort would be required to obtain an alpha value of 5% and a power of 80%: null proportion (p0) = 0.99, n = 368; p0 = 0.995, n = 736; p0 = 0.996, n = 921; p0 = 0.997, n = 1,228; p0 = 0.998, n = 1,843; p0 = 0.999, n = 3,688; p0 = 0.9995, n = 7,376. A sample size (n) of >10,000 would ensure that the analyses were sufficiently powered to provide a reliable estimate of assay specificity. For the cross-reactivity analyses, the aim was to evaluate ≥10 samples per cross-reacting factor.

16Determinations were performed as single measurements. For sensitivity and specificity, point estimates and 95% CIs were calculated using the Roche Diagnostics bioWARP tool (Roche Diagnostics GmbH, Penzberg, Germany). For contingency between Elecsys Anti-SARS-CoV-2 immunoassay and VSV-based pseudoneutralization assay results, PPA, PNA, and POA were calculated using the Westgard QC 2×2 Contingency Calculator (Westgard QC Inc., WI, USA).

17Qualified researchers may request access to individual-patient-level data through the clinical study data request platform (https://vivli.org/). Details on Roche's criteria for eligible studies are available at https://vivli.org/members/ourmembers/. Further details on Roche's Global Policy on the Sharing of Clinical Information and how to request access to related clinical study documents are available on the company website.

18A total of 496 samples from 102 patients with prior PCR-confirmed SARS-CoV-2 infection were included in the sensitivity analyses. All positive-sample donors (anonymized and aggregated data) were adults (mean [standard deviation {SD}] age, 66 [16.4] years) and had an average length of stay in hospital of 18 (SD, 14.9) days. Overall, 15.7% of the sample donors were admitted to an intensive care unit during the hospital stay. Sensitivity increased with time after PCR confirmation: 60.2% (95% CI, 52.3 to 67.8%) at 0 to 6 days, 85.3% (95% CI, 78.6 to 90.6%) at 7 to 13 days, and 99.5% (95% CI, 97.0 to 100.0%) at ≥14 days (Table 1; Fig. 1).

19Figure 2 shows Elecsys Anti-SARS-CoV-2 immunoassay results for 26 consecutive samples from five patients following recovery from PCR-confirmed SARS-CoV-2 infection. Patients had a range of SARS-CoV-2 symptoms, including one patient who reported no symptoms. In all patients with symptoms, cutoff index values were ≥20 following a negative PCR result and increased over time, out to day 40. In the patient with no symptoms, cutoff index values ranged from 0.99 to 1.97 following a negative PCR result.

20A total of 10,453 samples were included in the specificity analyses: 6,305 samples from routine diagnostic testing and 4,148 samples from blood donors. The overall specificity for the entire sample cohort was 99.80% (95% CI, 99.69 to 99.88%); specificity was 99.81% (95% CI, 99.67 to 99.90%) in routine diagnostic samples and 99.78% (95% CI, 99.59 to 99.90%) in blood donor samples (Table 2).

21Out of 792 samples with potential cross-reactivity, 4 samples were reactive with the Elecsys Anti-SARS-CoV-2 immunoassay: 1/85 (1.2%) acute cytomegalovirus infection samples, 2/105 (1.9%) acute Epstein-Barr virus infection samples, and 1/10 (10.0%) systemic lupus erythematosus samples (Table 3). Of the samples that were tested for potential cross-reactivity with common cold (n = 40) and coronavirus (n = 40) (229E, NL63, OC43, HKU1) panels, none were reactive. The overall specificity in this cohort was 99.5% (95% CI, 98.6 to 99.9%) (Table 3).

22In total, 47 samples were evaluated using the Elecsys Anti-SARS-CoV-2 immunoassay and a VSV-based pseudoneutralization assay for the purpose of method comparison. One sample was positive on the Elecsys Anti-SARS-CoV-2 immunoassay but indeterminate using both the VSV-based pseudoneutralization assay and in-house immunofluorescence assays, so it was excluded from the analysis. Of the remaining 46 samples, 38 tested positive on both the Elecsys Anti-SARS-CoV-2 immunoassay and the VSV-based pseudoneutralization assay (Table 4). Two samples were negative on both assays, with very low cutoff indexes observed for measurements taken using the Elecsys Anti-SARS-CoV-2 immunoassay (0.067 and 0.082). Six samples that tested positive using the VSV pseudoneutralization assay were negative on the Elecsys Anti-SARS-CoV-2 immunoassay; these samples had elevated cutoff indexes (0.206 to 0.659) compared with the two samples confirmed negative on both assays. When retested using an in-house immunofluorescence assay, two of these samples were found to be positive and four negative; all results were close to prespecified cutoffs.

23The PPA between the Elecsys Anti-SARS-CoV-2 immunoassay and the VSV pseudoneutralization assay was 86.4% (95% CI, 73.3 to 93.6%); the PNA was 100% (95% CI, 34.2 to 100%); and the POA was 87.0% (95% CI, 74.3 to 93.9%).

24The COVID-19 pandemic has resulted in significant mortality and morbidity and has created major challenges for governments and health care systems. Due to the novelty of the causative agent, SARS-CoV-2, and the rapidity with which it has spread, there is an urgent need for serological assays, which could be used to determine the seroprevalence in a given population and help identify a potential correlate for immunity secondary to exposure. This urgency and the lack of existing available tests prompted the U.S. Food and Drug Administration to allow manufacturers of serological assays for SARS-CoV-2 to bypass the normal approval process; as a result, the market has been inundated with tests, many of which have not been sufficiently reviewed by regulatory authorities nor their performance reported in the peer-reviewed literature. However, it is vital that any new test be adequately evaluated and validated to demonstrate that it is reliable and accurate for its intended purpose.

25To enable population screening to determine the level of exposure and to identify individuals who may be immune following infection, an ideal serological test for SARS-CoV-2 should meet several key requirements: (i) a high general specificity with a small CI, (ii) no cross-reactivity with other endemic coronaviruses, (iii) preference for the detection of mature antibodies to provide the potential for correlation with neutralizing activity, and (iv) a high sample throughput to meet the huge demand for testing. The first three requirements can be achieved through the application of an appropriate assay format and antigen selection, while the last requirement relies on a high-throughput platform and upscaling of laboratories. Although a high assay sensitivity is desirable, it is of secondary importance and should not be pursued at the expense of specificity for past infection, since a serological assay is less likely to be used for the diagnosis of active infection.

26The Elecsys Anti-SARS-CoV-2 immunoassay was specifically designed to meet these requirements. It utilizes a double-antigen sandwich test principle for the detection of high-affinity (i.e., late-onset/mature) antibodies to SARS-CoV-2. In the present study, the Elecsys Anti-SARS-CoV-2 immunoassay demonstrated an overall specificity of 99.80% (95% CI, 99.69 to 99.88%) in 10,453 residual samples from routine testing and blood donors. The sensitivity of the Elecsys Anti-SARS-CoV-2 immunoassay in samples from patients with prior PCR-confirmed SARS-CoV-2 infection increased with time after PCR confirmation, reaching 99.5% (95% CI, 97.0 to 100.0%) at ≥14 days. The performance of the Elecsys Anti-SARS-CoV-2 immunoassay is comparable to or better than that observed for other serological SARS-CoV-2 assays (specificity, 94.8 to 99.9%; sensitivity at ≥14 days post-PCR confirmation, 75.0 to 100.0%).

27The main risk with the use of serological SARS-CoV-2 assays for population screening is the possibility of false-positive results, which could lead to the erroneous assumption of past infection and subsequent putative immunity and thus could put the individual at risk of acquiring or transmitting infection. A very high specificity is crucial to reduce the rate of false-positive results, particularly in populations with low seroprevalence, where small differences in assay specificity can result in substantial differences in the positive predictive value (PPV). For example, if the seroprevalence of SARS-CoV-2 in a given population was 10%, a serological test with a sensitivity of 83.1% and a specificity of 98.3% would have a PPV of 84.2%; however, if the seroprevalence was 1%, the respective PPV would drop to 32.6%. Understanding this relationship between seroprevalence, specificity, and PPV is crucial for population screening using serological testing, and it has been suggested that assay specificity should be >99% (with a CI of 99.0 to 99.9%) to ensure a sufficient PPV in populations with low seroprevalence. Based on a sensitivity of 99.5% and a specificity of 99.8%, the PPV of the Elecsys Anti-SARS-CoV-2 immunoassay in populations of 1%, 5%, 10%, and 20% seroprevalence would be 83.4%, 96.3%, 98.2%, and 99.2%, respectively. A potential cause of false-positive results is cross-reactivity with other analytes. In the present study, only 4/792 samples containing potential cross-reacting analytes showed reactivity with the Elecsys Anti-SARS-CoV-2 immunoassay; importantly, no cross-reactivity was observed for the coronavirus panel (containing strains 229E, NL63, OC43, and HKU1). This resulted in an overall specificity of 99.5%.

28The present findings indicate that a positive test result from the Elecsys Anti-SARS-CoV-2 immunoassay correlates well with in vitro neutralization activity; good PPA (86.4%), PNA (100%), and POA (87.0%) were observed relative to results from a VSV-based pseudoneutralization assay. However, it should be noted that the PNA reported herein is based on only two isolates and thus requires further validation. These findings are in overall agreement with those of Müller et al., who compared four commercially available serological assays for SARS-CoV-2 (the Elecsys Anti-SARS-CoV-2 immunoassay, EuroImmun anti-SARS-CoV-2 IgA and IgG enzyme-linked immunosorbent assay [ELISA], Abbott SARS-CoV-2 IgG chemiluminescent microparticle immunoassay [CMIA], and Liaison SARS-CoV-2 S1/S2 IgG chemiluminescent immunoassay [CLIA]) with a neutralization assay and observed good PPA (65.4%), PNA (100%), and POA (78.6%) for the Elecsys Anti-SARS-CoV-2 immunoassay. Our results also align with those of Kohmer et al., who compared six commercially available serological assays for SARS-CoV-2 (the Elecsys Anti-SARS-CoV-2 immunoassay, EuroImmun anti-SARS-CoV-2 IgG ELISA, Abbott SARS-CoV-2 IgG CMIA, Liaison SARS-CoV-2 S1/S2 IgG CLIA, VirClia COVID-19 IgG Monotest CLIA, and Virotech SARS-CoV-2 IgG ELISA) with an in-house plaque reduction neutralization test and observed good PPA (75.6%), PNA (97.1%), and POA (84.8%). Nucleocapsid- and spike-based assays showed equal agreement with neutralization assays in each of these studies; however, it should be noted that the VSV-based pseudoneutralization assay utilized herein expresses only the SARS-CoV-2 spike protein and is therefore capable of capturing only spike-targeted antibodies. This should be taken into consideration when one is comparing nucleocapsid-based serological immunoassays, such as the Elecsys Anti-SARS-CoV-2 immunoassay, with a pseudoneutralization assay.

29Based on our understanding of other respiratory viruses, it has been suggested that the presence of anti-SARS-CoV-2 antibodies will provide some immunity, although it is unclear how long any protection would last after the initial infection. Assuming that anti-SARS-CoV-2 antibodies do offer some immunity from further infection, donated plasma from patients who have recovered from COVID-19 may represent a potential therapy by conferring immunity on the recipient. In this context, serological assays could play another important role in the identification of potential convalescent plasma donors, particularly individuals with very high anti-SARS-CoV-2 antibody titers. Recent studies suggest that both cell-mediated and humoral immune responses are likely to play a protective role in SARS-CoV-2 infection, and the spike and nucleocapsid antigens in particular have been shown to be highly immunogenic and abundantly expressed during infection. Antibodies targeting these proteins are formed as early as 9 days after symptom onset and have demonstrated a strong neutralizing response, suggesting that seroconversion may lead to immunity for a limited time after infection. However, further research is needed to demonstrate the correlation between the presence of anti-SARS-CoV-2 antibodies and neutralization, as well as, ultimately, the correlation with clinical immunity. Additional work is also required to establish whether there is a correlation between anti-SARS-CoV-2 antibody titers and disease severity and/or prognosis.

30A major strength of this study is the large number of seronegative samples (n = 10,453) used to determine the specificity of the Elecsys Anti-SARS-CoV-2 immunoassay, which ensures that the study is robustly powered and the reliability of the specificity point estimates is very high (as indicated by the small CIs). To our knowledge, this is one of the largest sample sizes used to date to evaluate the specificity of a serological assay for SARS-CoV-2. Another study strength is the inclusion of a high number of confounder samples in the cross-reactivity cohort, which comprised a total of 792 samples across 36 different indications. A limitation is that this was a single-center study, and our results should be confirmed by additional assessments at other study sites. Further clinical data on the samples were not available due to data regulations, and thus, it was not possible to analyze specific subcohorts according to age, disease severity, onset of symptoms, etc. In addition, a comparably small sample size was utilized to measure agreement between the Elecsys Anti-SARS-CoV-2 immunoassay and a pseudoneutralization assay; as such, these data would benefit from further evaluation and validation. We are confident that our results regarding the sensitivity of the Elecsys Anti-SARS-CoV-2 immunoassay and the time course of antibody response are of general value. However, the findings should be interpreted with some caution. Although the samples used in this study were not selected on the basis of patient criteria, the majority of samples were drawn from hospitalized patients and therefore probably represent more-severe cases of COVID-19. Furthermore, the availability of samples depended on the need for consequent routine clinical chemistry diagnosis after PCR confirmation of SARS-CoV-2 infection. Again, it can be assumed that extensive subsequent diagnosis was performed predominantly in cases with more-severe disease. For an asymptomatic patient assessed following recovery from PCR-confirmed SARS-CoV-2 infection in this study, cutoff index values remained close to the positive/negative threshold for anti-SARS-CoV-2 antibodies. Therefore, the validity of our findings in ambulatory settings or for patients with asymptomatic/mildly symptomatic SARS-CoV-2 infection has yet to be shown and requires further study.

31The Elecsys Anti-SARS-CoV-2 immunoassay demonstrated a very high specificity of 99.80% and a sensitivity of 99.5% for past infection in patients at ≥14 days after PCR confirmation, supporting its use as a potential tool for the identification of past exposure to SARS-CoV-2 infection. High assay specificity and sensitivity are crucial to ensure a high PPV for population screening, particularly in settings with low disease prevalence.