Accuracy of novel antigen rapid diagnostics for SARS-CoV-2: A living systematic review and meta-analysis

1As the COVID-19 pandemic continues around the globe, antigen rapid diagnostic tests (Ag-RDTs) for SARS-CoV-2 are seen as an important diagnostic tool to fight the virus’s spread. The number of Ag-RDTs on the market is increasing constantly. Initial data from independent evaluations suggest that the performance of SARS-CoV-2 Ag-RDTs may be lower than what is reported by the manufacturers. In addition, Ag-RDT accuracy seems to vary substantially between tests.

2With the increased availability of Ag-RDTs, an increasing number of independent validations have been published. Such evaluations differ widely in their quality, methods, and results, making it difficult to assess the true performance of the respective tests. To inform decision makers on the best choice of individual tests, an aggregated, widely available, and frequently updated assessment of the quality, performance, and independence of the data is urgently needed. While other systematic reviews have been published, they include data only up until November 2020, exclude preprints, or were industry sponsored. In addition, only 1 assessed the quality of studies in detail, with data up until November 2020.

3With our systematic review and meta-analysis, we aim to close this gap in the literature and link to a website (https://www.diagnosticsglobalhealth.org) that is regularly updated.

4We developed a study protocol following standard guidelines for systematic reviews, which is available in S1 Text. We also completed the PRISMA checklist (S1 PRISMA Checklist). Furthermore, we registered the review on PROSPERO (registration number: CRD42020225140).

5We performed a search of the databases PubMed, Web of Science, medRxiv, and bioRxiv using search terms that were developed with an experienced medical librarian (M. Grilli) using combinations of subject headings (when applicable) and text-words for the concepts of the search question. The main search terms were “Severe Acute Respiratory Syndrome Corona-virus 2,” “COVID-19,” “Betacoronavirus,” “Coronavirus,” and “Point of Care Testing.” The full list of search terms is available in S2 Text. We also searched the Foundation for Innovative New Diagnostics (FIND) website (https://www.finddx.org/sarscov2-eval-antigen/) for relevant studies manually. We performed the search up until 30 April 2021. No language restrictions were applied.

6We included studies evaluating the accuracy of commercially available Ag-RDTs to establish a diagnosis of SARS-CoV-2 infection, against reverse transcription polymerase chain reaction (RT-PCR) or cell culture as reference standard. We included all study populations irrespective of age, presence of symptoms, or study location. We considered cohort studies, nested cohort studies, case–control or cross-sectional studies, and randomized studies. We included both peer-reviewed publications and preprints.

7We excluded studies in which patients were tested for the purpose of monitoring or ending quarantine. Also, publications with a population size smaller than 10 were excluded. Although the size threshold of 10 is arbitrary, such small studies are more likely to give unreliable estimates of sensitivity and specificity.

8Ag-RDTs for SARS-CoV-2 aim to detect infection by recognizing viral proteins. Most Ag-RDTs use specific labeled antibodies attached to a nitrocellulose matrix strip, to capture the virus antigen. Successful binding of the antibodies to the antigen either is detected visually (through the appearance of a line on the matrix strip [lateral flow assay]) or requires a specific reader for fluorescence detection. Microfluidic enzyme-linked immunosorbent assays have also been developed. Ag-RDTs typically provide results within 10 to 30 minutes.

9Viral culture detects viable virus that is relevant for transmission but is available in research settings only. Since RT-PCR tests are more widely available and SARS-CoV-2 RNA (as reflected by RT-PCR cycle threshold [Ct] value) highly correlates with SARS-CoV-2 antigen quantities, we considered it an acceptable reference standard for the purposes of this systematic review. It is of note that there is currently no international standard for the classification of viral load available.

10Two reviewers (LEB and CE, LEB and SS, or LEB and MB) reviewed the titles and abstracts of all publications identified by the search algorithm independently, followed by a full-text review for those eligible, to select the articles for inclusion in the systematic review. Any disputes were solved by discussion or by a third reviewer (CMD).

11A full list of the parameters extracted is included in S1 Table, and the data extraction file is available at https://zenodo.org/record/4924035#.YOlzWS223RZ. Studies that assessed multiple Ag-RDTs or presented results based on differing parameters (e.g., various sample types) were considered as individual datasets.

12At first, 4 authors (SK, CE, SS, and MB) extracted 5 randomly selected papers in parallel to align data extraction methods. Afterwards, data extraction and the assessment of methodological quality and independence from test manufacturers (see below) was performed by 1 author per paper (SK, CE, SS, or MB) and controlled by a second (LEB, SK, SS, or MB). Any differences were resolved by discussion or by consulting a third author (CMD).

13We differentiated between clinical accuracy studies (performed on clinical samples) and analytical accuracy studies (performed on spiked samples with a known quantity of virus). Analytical accuracy studies can differ widely in methodology, impeding an aggregation of their results. Thus, while we extracted the data for both kinds of studies, we only considered data from clinical accuracy studies as eligible for the meta-analysis. Separately, we summarized the results of analytical studies and compared them with the results of the meta-analysis for individual tests.

14The quality of the clinical accuracy studies was assessed by applying the QUADAS-2 tool. The tool evaluates 4 domains: patient selection, index test, reference standard, and flow and timing. For each domain, the risk of bias is analyzed using different signaling questions. Beyond the risk of bias, the tool also evaluates the applicability of each included study to the research question for every domain. The QUADAS-2 tool was adjusted to the needs of this review and can be found in S3 Text.

15We examined whether a study received financial support from a test manufacturer (including the free provision of Ag-RDTs), whether any study author was affiliated with a test manufacturer, and whether a respective conflict of interest was declared. Studies were judged not to be independent from the test manufacturer if at least 1 of these aspects was present; otherwise, they were considered to be independent.

16We extracted raw data from the studies and recalculated performance estimates where possible based on the extracted data. The raw data can be found in S2 Table. We prepared forest plots for the sensitivity and specificity of each test and visually evaluated the heterogeneity between studies. If 4 or more datasets were available with at least 20 positive RT-PCR samples per dataset for a predefined analysis, a meta-analysis was performed. We report point estimates of sensitivity and specificity for SARS-CoV-2 detection compared to the reference standard along with 95% confidence intervals (CIs) using a bivariate model (implemented with the “reitsma” command from the R package “mada,” version 0.5.10). When there were fewer than 4 studies for an index test, only a descriptive analysis was performed, and accuracy ranges are reported. In subgroup analyses where papers presented data only on sensitivity, a univariate random-effects inverse variance meta-analysis was performed (using the “metagen” command from the R package “meta,” version 4.11–0). We predefined subgroups for meta-analysis based on the following characteristics: Ct value range, sampling and testing procedure in accordance with manufacturer’s instructions as detailed in the instructions for use (IFU) (henceforth called IFU-conforming) versus not IFU-conforming, age (<18 versus ≥18 years), sample type, presence or absence of symptoms, symptom duration (<7 days versus ≥7 days), viral load, and type of RT-PCR used.

17In an effort to use as much of the heterogeneous data as possible, the cutoffs for the Ct value groups were relaxed by 2–3 points within each range. The <20 group included values reported up to ≤20, the <25 group included values reported as ≤24 or <25 or 20–25, and the <30 group included values from ≤29 to ≤33 and 25–30. The ≥25 group included values reported as ≥25 or 25–30, and the ≥30 group included values from ≥30 to ≥35. For the same reason, when categorizing by age, the age group <18 years (children) included samples from persons whose age was reported as <16 or <18 years, whereas the age group ≥18 years (adults) included samples from persons whose age was reported as ≥16 years or ≥18.

18For categorization by sample type, we assessed (1) nasopharyngeal (NP) alone or combined with other (e.g., oropharyngeal [OP]), (2) OP alone, (3) anterior nasal (AN) or mid-turbinate (MT), (4) a combination of bronchoalveolar lavage and throat wash (BAL/TW), or (5) saliva. Analyses were preformed using R 4.0.3 (R Foundation for Statistical Computing, Vienna, Austria).

19We aimed to do meta-regression to examine the impact of covariates including symptom duration and Ct value range. We also performed the Deeks test for funnel-plot asymmetry as recommended to investigate publication bias for diagnostic test accuracy meta-analyses (using the “midas” command in Stata, version 15); a p-value < 0.10 for the slope coefficient indicates significant asymmetry.

20Two types of sensitivity analyses were planned: estimation of sensitivity and specificity excluding case–control studies, and estimation of sensitivity and specificity excluding non-peer-reviewed studies. We compared the results of each sensitivity analysis against the overall results to assess the potential bias introduced by considering case–control studies and non-peer-reviewed studies.

21The systematic search resulted in 14,254 articles. After removing duplicates, 8,921 articles were screened, and 266 papers were considered eligible for full-text review. Of these, 148 were excluded because they did not present primary data or the Ag-RDT was not commercially available, leaving 133 studies to be included in the systematic review (Fig 1).

22At the end of the data extraction process, 37 studies were still in preprint form. All studies were written in English, except for 2 in Spanish. Out of the 133 studies, 9 reported analytical accuracy, and the remaining 124 reported clinical accuracy.

23The clinical accuracy studies were divided into 214 datasets, while the 9 analytical accuracy studies accounted for 63 datasets. A total of 61 different Ag-RDTs were evaluated (48 lateral flow with visual readout and 12 requiring an automated reader), with 56 being assessed in a clinical accuracy study. Thirty-nine studies reported data for more than 1 test, and 19 of these studies conducted a head-to-head assessment, i.e., testing at least 2 Ag-RDTs on the same sample or participant. The reference method was RT-PCR in all except 1 study, which used viral culture.

24The most common reasons for testing were the occurrence of symptoms (55/19.9% of datasets), screening independent of symptoms (19/6.9%), and close contact with a SARS-CoV-2 confirmed case (10/3.6%). In 79 (28.6%) of the datasets, persons were tested due to more than 1 of these reasons, and for 163 datasets (59.1%), the reason for testing was unclear.

25In total, 113,242 Ag-RDTs were performed, 112,323 (99.2%) in clinical accuracy studies and 919 (0.8%) in analytical accuracy studies. In the clinical accuracy studies, the mean number of samples per study was 525 (range 16 to 6,954). Only 4,752 (4.2%) tests were performed on pediatric (age group <18 years) samples, and 21,351 (18.9%) on samples from adults (age group ≥18 years). For the remaining 87,139 (76.9%) samples, the age of the persons tested was not specified. Symptomatic patients comprised 36,981 (32.7%) samples; 32,799 (29.0%) samples originated from asymptomatic patients, and for 42,462 (38.4%) samples, the patient’s symptom status could not be identified. The most common sample type evaluated was NP and mixed NP/OP (67,036 samples, 59.2%), followed by AN/MT (27,045 samples, 23.9%). There was substantially less testing done for the other sample types, with 6,254 (5.5%) tests done from OP samples, 1,351 (1.2%) from saliva, and 219 (0.2%) from BAL/TW, and for 11,337 (10.0%) tests, we could not identify the type of sample.

26Of the datasets assessing clinical accuracy, 89 (41.6%) involved testing according to the manufacturers’ recommendations (i.e., IFU-conforming), while 100 (46.7%) were not IFU-conforming, and for 25 (11.7%) it was unclear. The most common deviations from the IFU were (1) use of samples that were prediluted in transport media not recommended by the manufacturer (80 datasets; 7 unclear), (2) use of banked samples (60 datasets; 14 unclear), and (3) use of a sample type that was not recommended for the Ag-RDT (17 datasets; 8 unclear).

27A summary of the tests evaluated in clinical accuracy studies, including study identification, sample size, sample type, sample condition, and IFU conformity, can be found in Table 1. The Panbio test by Abbott (Germany; henceforth called Panbio) was reported the most frequently, with 39 (18.2%) datasets and 28,089 (25.0%) tests, while the Standard Q test by SD Biosensor (South Korea; distributed in Europe by Roche, Germany; henceforth called Standard Q) was assessed in 37 (17.3%) datasets, with 16,820 (15.0%) tests performed. Detailed results for each clinical accuracy study are available in S1 Fig.

28The findings on study quality using the QUADAS-2 tool are presented in Figs 2 and 3. In 190 (88.8%) datasets a relevant patient population was assessed. However, for only 44 (20.6%) of the datasets was patient selection considered representative of the setting and population chosen (i.e., they avoided inappropriate exclusions and a case–control design, and enrollment occurred consecutively or randomly).

29The conduct and interpretation of the index tests was considered to have low risk for introduction of bias in 113 (52.8%) datasets (through, e.g., appropriate blinding of persons interpreting the visual readout). However, for 99 (46.3%) datasets, sufficient information to clearly judge the risk of bias was not provided. In only 89 (41.6%) datasets were the Ag-RDTs performed according to IFU, while 100 (46.7%) were not IFU-conforming, potentially impacting the diagnostic accuracy (for 25 [11.7%] datasets the IFU status was unclear).

30In 81 (37.9%) datasets, the reference standard was performed before the Ag-RDT, or the operator conducting the reference standard was blinded to the Ag-RDT results, resulting in a low risk of bias. In almost all other datasets (132/61.7%), this risk could not be assessed due to missing data. The applicability of the reference test was judged to be of low concern for all datasets, as cell culture and RT-PCR are expected to adequately define the target condition.

31In 209 (97.7%) datasets, the sample for the index test and reference test were obtained at the same time, while this was unclear in 5 (2.3%) datasets. All samples included in a dataset were subjected to the same type of RT-PCR in 145 (67.8%) datasets, while different types of RT-PCR were used within the same dataset in 50 (23.4%) datasets. For 19 (8.9%) datasets, it was unclear. Furthermore, for 11 (5.1%) datasets, there was a concern that not all selected patients were included in the analysis.

32Finally, 32 (24.1%) of the studies received financial support from the Ag-RDT manufacturer, and in another 9 (6.8%) studies, employment of the authors by the manufacturer of the Ag-RDT studied was indicated. Overall, a competing interest was found in 33 (24.8%) of the studies.

33Out of 214 clinical datasets (from 124 studies), 20 were excluded from the meta-analysis because they included fewer than 20 RT-PCR positive samples. A further 21 datasets were missing either sensitivity or specificity and were only considered for univariate analyses. Across the remaining 173 datasets, including any test and type of sample, the pooled sensitivity and specificity were 71.2% (95% CI 68.2% to 74.0%) and 98.9% (95% CI 98.6% to 99.1%), respectively. If testing was performed in conformity with IFU, sensitivity increased to 76.3% (95% CI 73.1% to 79.2%), while non-IFU-conforming testing had a sensitivity of 65.9% (95% CI 60.6% to 70.8%). Pooled specificity was similar in both groups (99.1% [95% CI 98.8–99.4%] and 98.3% [95% CI 97.7% to 98.8%], respectively).

34Based on 119 datasets with 71,424 tests performed, we were able to perform bivariate meta-analysis of the sensitivity and specificity for 12 different Ag-RDTs (Fig 4). Across these, the pooled estimates of sensitivity and specificity on all samples were 72.1% (95% CI 68.8% to 75.3%) and 99.0% (95% CI 98.7% to 99.2%), respectively, which were very similar to the overall pooled estimates across all meta-analyzed datasets (71.2% and 98.9%, respectively, above).

35The highest pooled sensitivity was found for the SARS-CoV-2 Antigen Test by LumiraDx (UK; henceforth called LumiraDx) and the Lumipulse G SARS-CoV-2 Ag by Fujirebio (Japan; henceforth called Lumipulse G), with 88.2% (95% CI 59.0% to 97.5%) and 87.2% (95% CI 78.0% to 92.9%), respectively. The Sofia SARS Antigen FIA by Quidel (California, US; henceforth called Sofia) had a pooled sensitivity of 77.4% (95% CI 74.2% to 80.3%). Of the non-instrument tests, the Standard Q and the Standard Q nasal test by SD Biosensor (South Korea; distributed in Europe by Roche, Germany; henceforth called Standard Q nasal) performed best, with a pooled sensitivity of 74.9% (95% CI 69.3% to 79.7%) and 80.2% (95% CI 70.3% to 87.4%), respectively. The pooled sensitivity for Panbio was 71.8% (95% CI 65.4% to 77.5%). Of all Ag-RDTs, the COVID-19 Ag Respi-Strip by Coris BioConcept (Belgium; henceforth called Coris) had the lowest pooled sensitivity, 40.0% (95% CI 28.7% to 52.4%).

36The pooled specificity was above 98% for all of the tests, except for the Standard F by SD Biosensor (South Korea) and Lumipulse G, with specificities of 97.7% (95% CI 96.6% to 98.5%) and 96.7% (95% CI 88.6% to 99.1%), respectively. Hierarchical summary receiver operating characteristic values for Standard Q and LumiraDx are available in S2 Fig.

37Three Ag-RDTs did not have sufficient data to allow for a bivariate meta-analysis, so a univariate analysis was conducted (Fig 5). For the INNOVA SARS-CoV-2 Antigen Rapid Qualitative Test by Innova Medical Group (California, US), this resulted in a pooled sensitivity and specificity of 76.1% (95% CI 68.1% to 84.1%) and 99.4% (95% CI 98.7% to 100%), respectively. For the NADAL by nal von minden (Germany) and the COVID-19 Rapid Antigen Visual Read by SureScreen Diagnostics (UK), sufficient data were available to analyze only sensitivity, resulting in pooled sensitivity estimates of 58.4% (95% CI 29.2% to 87.6%) and 58.0% (95% CI 38.3% to 77.6%), respectively.

38The remaining 35 Ag-RDTs did not present sufficient data for univariate or bivariate meta-analysis. However, 9/35 had results presented in more than 1 dataset, and these are summarized in Table 2. Herein, the widest ranges of sensitivity were found for the ESPLINE SARS-CoV-2 by Fujirebio (Japan), with sensitivity reported between 8.1% and 80.7%, and the RIDA QUICK SARS-CoV-2 Antigen by R-Biopharm (Germany), with sensitivity between 39.2% and 77.6%, both with 3 datasets each. In contrast, 2 other tests with 2 datasets each showed the least variability in sensitivity: The Zhuhai Encode Medical Engineering SARS-CoV-2 Antigen Rapid Test (China) reported sensitivity between 74.0% and 74.4%, and the COVID-19 Rapid Antigen Fluorescent by SureScreen Diagnostics (UK) reported sensitivity between 60.3% and 69.0%. However, for both tests, both datasets originated from the same studies. Overall, the lowest sensitivity range was reported for the SARS-CoV-2 Antigen Rapid Test by MEDsan (Germany): 36.5% to 45.2% across 2 datasets. The specificity ranges were above 96% for most of the tests. A notable outlier was the 2019-nCov Antigen Rapid Test Kit by Shenzhen Bioeasy Biotechnology (China; henceforth called Bioeasy), reporting the worst, with a specificity as low as 85.6% in 1 study. Forest plots for the datasets for each Ag-RDT are provided in S3 Fig. The remaining 26 Ag-RDTs that were evaluated in 1 dataset only are included in Table 1 S3 Fig.

39In total, 16 studies, accounting for 53 datasets, conducted head-to-head clinical accuracy evaluations of different tests using the same samples from the same participants. These datasets have underlined sample sizes in Table 1; 15 such studies included more than 100 samples, and 1 study included too few samples to draw clear conclusions. Four studies performed their head-to-head evaluation as per manufacturers’ instructions and on symptomatic patients. Across 3 of them, Standard Q (sensitivity 73.2% to 91.2%) and Standard Q nasal (sensitivity 82.5% to 91.2%) showed a similar range of sensitivity. The fourth reported a sensitivity of 56.4% (95% CI 44.7% to 67.6%) for the Biocredit Covid-19 Ag by RapiGEN (South Korea; henceforth called Rapigen) and 52.6% (95% CI 40.9% to 64.0%) for the SGTi-flex COVID-19 Ag by Sugentech (South Korea).

40All other head-to-head comparisons were not IFU-conforming. In one of these, the Rapid COVID-19 Ag Test by Healgen (sensitivity 77.1%) performed better than Standard Q and Panbio (sensitivity 69.8% and 67.7%, respectively). In contrast to the overall findings of the meta-analysis above, 2 other head-to-head studies found that both Standard Q (sensitivity 43.6% and 49.4%) and Panbio (sensitivity 38.6% and 44.6%) had lower performance than the CLINITEST Rapid COVID-19 Antigen Test by Siemens Healthineers (Germany; henceforth called Clinitest), with reported sensitivity of 51.5% and 54.9%. However, another study found both Standard Q and Panbio (sensitivity 81.0% and 82.9%, respectively) to have a higher accuracy than Sofia (sensitivity 80.4%).

41The results are presented in Figs 6–10. Detailed results for the subgroup analyses are available in S4–S9 Figs.

42High sensitivity was achieved for Ct value < 20, at 96.5% (95% CI 92.6% to 98.4%). The pooled sensitivity for Ct value < 25 was markedly better, at 95.8% (95% CI 92.3% to 97.8%), compared to the group with Ct value ≥ 25, at 50.7% (95% CI 35.6% to 65.8%). A similar pattern was observed when the Ct values were analyzed using the cutoffs <30 and ≥30, resulting in a sensitivity of 79.9% (95% CI 70.3% to 86.9%) and 20.9% (95% CI 12.5% to 32.8%), respectively (Fig 6).

43In addition, it was possible to meta-analyze test-specific pooled sensitivity for Panbio: 97.7% sensitivity (95% CI 95.3% to 98.9%) for Ct value < 20, 95.8% (95% CI 92.3% to 97.8%) for Ct value < 25, and 83.4% (95% CI 69.1% to 91.9%) for Ct value < 30. Sensitivity was 61.2% (95% CI 38.8% to 79.7%) for Ct value ≥ 25 and 30.5% (95% CI 16.0% to 50.4%) for Ct value ≥ 30. For the other Ag-RDTs only limited data were available, which are presented in S5 Fig.

44The summary results are presented in Fig 7. When assessing only studies with IFU-conforming testing, pooled sensitivity from 81 datasets with 49,643 samples was 76.3% (95% CI 73.1% to 79.2%). When non-IFU-conforming sampling (75 datasets, 31,416 samples) was performed, sensitivity decreased to 65.9% (95% CI 60.6% to 70.8%).

45For 5 tests it was possible to calculate pooled sensitivity estimates including only datasets with IFU-conforming testing: Panbio (sensitivity 76.5% [95% CI 69.5% to 82.3%]; 17 datasets, 12,856 samples), Standard Q (sensitivity 79.3% [95% CI 73.5% to 84.1%]; 15 datasets, 6,584 samples), BinaxNOW (sensitivity 61.8% [95% CI 48.0% to 74.0%]; 4 datasets, 8,163 samples), Rapigen (sensitivity 67.1% [95% CI 50.4% to 80.4%]; 4 datasets, 1,934 samples), and Standard Q nasal (sensitivity 83.8% [95% CI 77.8% to 88.4%]; 5 datasets, 683 samples). Specificity was above 98.6% for all tests.

46In contrast, when the Panbio (14 datasets, 9,233 samples) and Standard Q (14 datasets, 4,714 samples) tests were not performed according to IFU, pooled sensitivity decreased to 64.3% (95% CI 50.9% to 75.8%) and 67.4% (95% CI 57.2% to 76.2%), respectively.

47Most datasets evaluated NP or combined NP/OP swabs (122 datasets and 59,810 samples) as the sample type for the Ag-RDT. NP or combined NP/OP swabs achieved a pooled sensitivity of 71.6% (95% CI 68.1% to 74.9%). Datasets that used AN/MT swabs for Ag-RDTs (32 datasets and 25,814 samples) showed a summary estimate for sensitivity of 75.5% (95% CI 70.4% to 79.9%). This was confirmed by 2 studies that reported direct head-to-head comparison of NP and MT samples from the same participants using the same Ag-RDT (Standard Q), where the 2 sample types showed equivalent performance. Analysis of performance with an OP swab (7 datasets, 5,165 samples) showed a pooled sensitivity of only 53.1% (95% CI 40.9% to 65.0%). Saliva swabs (4 datasets, 1,088 samples) showed the lowest pooled sensitivity, at only 37.9% (95% CI 11.8% to 73.5%) (Fig 8).

48We were not able to perform a subgroup meta-analysis for BAL/TW due to insufficient data: There was only 1 study with 73 samples evaluating Rapigen, Panbio, and Standard Q. However, BAL/TW would in any case be considered an off-label use.

49Within the datasets possible to meta-analyze, 17,964 (54.1%) samples were from symptomatic, and 15,228 (45.9%) from asymptomatic, patients. The pooled sensitivity for symptomatic patients was markedly different from that of asymptomatic patients: 76.7% (95% CI 70.6% to 81.9%) versus 52.5% (95% CI 43.7% to 61.1%). Specificity was 99% for both groups (Fig 9). Median Ct values differed in symptomatic and asymptomatic patients. For those studies where it was possible to extract a median Ct value, it ranged from 20.5 to 27.0 in symptomatic patients and from 27.2 to 30.5 in asymptomatic patients.

50Data were analyzed for 5,538 patients with symptoms less than 7 days, but very limited data were available for patients with symptoms ≥7 days (397 patients). The pooled sensitivity for patients with onset of symptoms <7 days was 83.8% (95% CI 76.3% to 89.2%), which is markedly higher than the 61.5% (95% CI 52.2% to 70.0%) sensitivity for individuals tested ≥7 days from onset of symptoms (Fig 9).

51For adult patients (age ≥ 18 years), it was possible to pool estimates across 3,837 samples, whereas the pediatric group (age < 18 years) included 7,326 samples. Sensitivity and specificity were 64.3% (95% CI 54.7% to 72.9%) and 99.4% (95% CI 98.9% to 99.7%), respectively, in mostly symptomatic patients aged <18 years. In patients aged ≥18 years, sensitivity increased to 74.8% (95% CI 66.5% to 81.6%), while the specificity was similar (98.7%, 95% CI 97.2% to 99.4%) (Fig 10).

52We were not able to perform a meta-analysis for the subgroups by type of RT-PCR or viral load (viral copies/mL) due to insufficient data.

53In 152 (71.0%) of the datasets only 1 type of RT-PCR was used, whereas 37 (17.3%) of the datasets tested samples in the same dataset using different RT-PCR methods. For 25 (11.7%) of the datasets, the type of RT-PCR was not reported. The Cobas SARS-CoV-2 Test from Roche (Germany) was used most frequently, in 63 (29.4%) of the datasets, followed by the Allplex 2019-nCoV Assay from Seegene in 41 (19.2%) and the SARS-CoV-2 assay from TaqPath in 20 (9.3%) of the datasets.

54Median sensitivity was 72.4% (range 46.9% to 100%) in samples with viral load > 5 log10 copies/mL, 97.8% (range 71.4% to 100%) for >6 log10 copies/mL, and 100% (range 93.8% to 100%) for >7 log10 copies/mL, showing that the sensitivity increases with increasing viral load.

55We were not able to perform a meta-regression due to the considerable heterogeneity in reporting subgroups, which resulted in too few studies with sufficient data for comparison.

56The result of the Deeks test (p = 0.001) shows significant asymmetry in the funnel plot for all datasets with complete results. This indicates there may be publication bias from studies with small sample sizes. The funnel plot is presented in S10 Fig.

57The 9 analytical studies were divided into 63 datasets, evaluating 23 different Ag-RDTs. Only 7 studies reported a sample size, for which 833 (90.6%) samples originated from NP swabs, while for 86 (9.4%) the sample type was unclear. One of the 2 studies not reporting sample size used saliva samples, while the other used the sample type specified in the respective Ag-RDT’s IFU.

58Overall, the reported analytical sensitivity (limit of detection [LOD]) in the studies resembled the results of the meta-analysis presented above. Rapigen (LOD, in log10 copies per swab: 10.2) and Coris (LOD 7.46) were found to perform worse than Panbio (LOD 6.6 to 6.1) and Standard Q (LOD 6.8 to 6.0), whereas Clinitest (LOD 6.0) and BinaxNOW by Abbott (LOD 4.6 to 4.9) performed better. Similar results were found in another study, where Standard Q showed the lowest LOD (detecting virus up to what is an equivalent Ct value of 26.3 to 28.7), compared to that of Rapigen and Coris (detecting virus up to what is an equivalent Ct value of only 18.4 for both). However, another study found Panbio, Standard Q, Coris, and BinaxNOW to have a similar LOD values of 5.0 × 103 plaque forming units (PFU)/mL, but the ESPLINE SARS-CoV-2 by Fujirebio (Japan), the COVID-19 Rapid Antigen Test by Mologic (UK), and the Sure Status COVID-19 Antigen Card Test by Premier Medical Corporation (India) performed markedly better (LOD 2.5 × 102 to 5.0 × 102 PFU/mL). An overview of all LOD values reported in the studies can be found in S3 Table.

59When the datasets from case–control studies (25/173) were excluded, the estimated sensitivity did not differ greatly, with a value of 70.9% (95% CI 67.7% to 73.9%), compared to 71.2% (95% CI 68.2% to 74.0%) in the overall analysis, with no change in pooled specificity. When the datasets from preprints (64/173) were excluded, sensitivity decreased slightly, to 67.2% (95% CI 62.9% to 71.3%), compared to the overall analysis.

60In this comprehensive systematic review and meta-analysis, we have summarized the data of 133 studies evaluating the accuracy of 61 different Ag-RDTs. Across all meta-analyzed samples, our results show a pooled sensitivity and specificity of 71.2% (95% CI 68.2% to 74.0%) and 98.9% (95% CI 98.6% to 99.1%), respectively. Over half of the studies did not perform the Ag-RDT in accordance with the test manufacturers’ recommendation, or the performance was unknown, which negatively impacted the sensitivity. When we considered only IFU-conforming studies, the sensitivity increased to 76.3% (95% CI 73.1% to 79.2%). While we found the sensitivity to vary across specific tests, the specificity was consistently high.

61The 2 Ag-RDTs that have been approved through the WHO emergency use listing procedure, Abbott Panbio and SD Biosensor Standard Q (distributed by Roche in Europe), have not only drawn the largest research interest, but also perform at or above average when their pooled accuracy is compared to that of all Ag-RDTs (sensitivity of 71.8% for Panbio and 74.9% for Standard Q). Standard Q nasal demonstrated an even higher pooled sensitivity (80.2% compared to the NP test), although this is likely due to variability in the populations tested, as head-to-head performance showed a comparable sensitivity. Three other Ag-RDTs showed an even higher accuracy, with sensitivities ranging from 77.4% to 88.2% (namely Sofia, Lumipulse G, and LumiraDx), but were only assessed on relatively small samples sizes (ranging from 1,373 to 3,532), and all required an instrument/reader.

62Not surprisingly, lower Ct values, the RT-PCR semi-quantitative correlate for high virus concentration, resulted in significantly higher Ag-RDT sensitivity than higher Ct values (pooled sensitivity 96.5% and 95.8% for Ct value < 20 and <25, respectively, versus 50.7% and 20.9% for Ct value ≥ 25 and ≥30, respectively). This confirms prior data that suggested that antigen concentrations and Ct values were highly correlated in NP samples. Ag-RDTs also showed higher sensitivity in patients within 7 days after symptom onset compared to patients later in the course of the disease (pooled sensitivity 83.8% versus 61.5%), which is to be expected given that samples from patients within the first week after symptom onset have been shown to contain the highest virus concentrations. In line with this, studies reporting an unexpectedly low overall sensitivity either shared a small population size with an on average high Ct value or performed the Ag-RDT not as per IFU, e.g., using saliva or prediluted samples. In contrast, studies with an unusually high Ag-RDT sensitivity were based on study populations with a low median Ct value, between 18 and 22.

63Our analysis also found that the accuracy of Ag-RDTs is substantially higher in symptomatic patients than in asymptomatic patients (pooled sensitivity 76.7% versus 52.5%). This is not surprising as studies that enrolled symptomatic patients showed a lower range of median Ct values (i.e., higher viral load) than studies enrolling asymptomatic patients. Given that other studies found symptomatic and asymptomatic patients to have comparable viral loads, the differences found in our analysis are likely explained by the varied time in the course of the disease at which testing is performed in asymptomatic patients presenting for one-time screening testing. Because symptoms start in the early phase of the disease, when viral load is still high, studies testing only symptomatic patients have a higher chance of including patients with high viral loads. In contrast, study populations drawn from only asymptomatic patients have a higher chance of including patients at any point of disease (i.e., including late in disease, when PCR is still positive, but viable virus is rapidly decreasing).

64With regards to the sampling and testing procedure, we found Ag-RDTs to perform similarly across upper respiratory swab samples (e.g., NP and AN/MT), particularly when considering the most reliable comparisons from head-to-head studies.

65Similar to previous assessment, the methodological quality of the included studies revealed a very heterogenous picture. In the future, aligning the design of clinical accuracy studies with common agreed-upon minimal specifications (e.g., by WHO or the European Centre for Disease Control and Prevention) and reporting the results in a standardized way would improve data quality and comparability.

66The main strengths of our study lie in its comprehensive approach and continuous updates. By linking this review to our website, https://www.diagnosticsglobalhealth.org, we strive to equip decision makers with the latest research findings on Ag-RDTs for SARS-CoV-2 and, to the best of our knowledge, are the first in doing so. At least once per week the website is updated by continuing the literature search and process described above. We plan to update the meta-analysis on a monthly basis and publish it on the website. Furthermore, our study used rigorous methods as both the study selection and data extraction were performed by one author and independently validated by a second, we conducted blinded pilot extractions before of the actual data extraction, and we prepared a detailed interpretation guide for the QUADAS-2 tool.

67The study may be limited by the inclusion of both preprints and peer-reviewed literature, which could affect the quality of the data extracted. However, we aimed to balance this potential effect by applying a thorough assessment of all clinical studies included, utilizing the QUADAS-2 tool, and performing a sensitivity analysis excluding preprint manuscripts. In addition, the studies included in our analysis varied widely in the reported range of viral loads, limiting the comparability of their results. To control for this, we analyzed the Ag-RDTs’ performance at different levels of viral load. Finally, even though we are aware that further data exist from other sources, for example from governmental research institutes, such data could not be included because sufficiently detailed descriptions of the methods and results are not publicly available.

68In summary, it can be concluded that there are Ag-RDTs available that have high sensitivity for the detection of a SARS-CoV-2 infection—particularly when performed in the first week of illness, when viral load is high—and excellent specificity. However, our analysis also highlights the variability in results between tests (which is not reflected in the manufacturer-reported data), indicating the need for independent validations. Furthermore, the analysis highlights the importance of performing tests in accordance with the manufacturers’ recommended procedures, and in alignment with standard diagnostic evaluation and reporting guidelines. The accuracy achievable by the best-performing Ag-RDTs, combined with the rapid turnaround time compared to RT-PCR, suggests that these tests could have a significant impact on the pandemic if applied in thoughtful testing and screening strategies.