Medical masks vs N95 respirators for preventing COVID‐19 in healthcare workers: A systematic review and meta‐analysis of randomized trials

1Novel 2019 coronavirus disease (COVID‐19) was declared a pandemic by the World Health Organization (WHO) on March 11, 2020, after the identification of >118 000 cases in 114 countries. 1 As government officials and public health stakeholders implement measures to slow the spread of SARS‐CoV‐2, healthcare workers treating COVID‐19 patients are among those at highest risk of infection. During the severe acute respiratory disease (SARS) pandemic in 2003, healthcare workers made up 21% (1706/8096) of global cases. 2 An early report from a single‐center case series of 138 hospitalized patients for COVID‐19 in Wuhan, China in January 2020 found that 29% (40/138) were healthcare workers that had been infected in hospital. 3 As of February 11, 2020, China's Infectious Disease Information System has reported COVID‐19 in 1716 healthcare workers. 4

2Although the transmission of COVID‐19 is not yet fully understood, it is believed to be mainly through large respiratory droplets. 5 For aerosol‐generating procedures, such as intubation or bronchoscopy, there is consensus that N95 respirators offer better protection than medical masks. 6, 7, 8, 9 N95 respirators are designed to minimize facial seal leakage because of tight fit and prevent inhalation of small airborne particles. They also are required to pass filtration tests. In contrast, medical masks (also known as surgical masks) are loose fitting, provide barrier protection against large droplets and prevent hand‐to‐face contact. 10

3Globally, current recommendations to protect healthcare workers against COVID‐19 for non–aerosol‐generating care are conflicting. 6, 7, 8, 9 For example, the U.S Centers for Disease Control and Prevention (CDC) and European Centre for Disease and Prevention (ECDC) recommend the N95 respirator for non–aerosol‐generating routine care of patients with COVID‐19, 6, 7 while the World Health Organization and the Public Health Agency of Canada recommend medical masks. 8, 9

4Shortages of personal protective equipment for healthcare workers, including medical masks and N95 respirators, have been widely reported in this pandemic. 11 A shortage of N95 respirators for aerosol‐generating procedures, where the risk to healthcare workers is high, is of particular concern. This, along with conflicting recommendations, warrants an update of previous systematic reviews (where the last search was conducted in 2015). 12, 13 Evidence to support similar relative effectiveness of medical masks compared with N95 respirators might help preserve stockpiles of N95 respirators for aerosol‐generating procedures. On the other hand, if N95 respirators are clearly more effective, then their use for non–aerosol‐generating procedures should be universally recommended. We conducted an updated systematic review and meta‐analysis to help answer this question.

5We adhered to the PRISMA statement when reporting of this review (Appendix S1). 14

6We adapted search strategies published by Smith et al by removing terms related to surrogate exposure studies (ie, simulations and experiments involving manikins) and applying database‐specific randomized controlled trial (RCT) filters (Appendix S2A‐C). 12, 15 We searched MEDLINE (OVID interface, Epub Ahead of Print, In‐Process & Other Non‐Indexed Citations, 1946 to Present), Embase (OVID interface, 1974 to Present) and the Cochrane Central Register of Controlled Trials (CENTRAL) from January 1, 2014, to March 9, 2020, for English‐language studies to update their search completed on December 9, 2014. 12 Two reviewers independently and in duplicate screened titles, abstracts and full‐texts of records identified by our searches. Consensus was reached through discussion among the review pair when necessary.

7We included RCTs that met all the following criteria: the design was an RCT including cluster randomized trials; the intervention was medical masks (defined surgical, procedural, isolation, laser, fluid resistant or face masks certified for use as a medical device) compared with N95 respirators (defined as respirators were N95 filtering face piece respirators certified by the National Institute for Occupational Safety and Health (NIOSH) and European standard filtering facepiece (FFP2) respirators) 12; the population was healthcare workers (defined as workers in a healthcare setting that could be exposed to a patient with acute respiratory illness) and reported on any of the following outcomes: viral respiratory infection laboratory confirmed by PCR, serology, or viral culture (our primary outcome), laboratory‐confirmed coronavirus infection, laboratory‐confirmed influenza infection, influenza‐like illness, clinical respiratory illness, or workplace absenteeism.

8A single reviewer extracted data on study characteristics, participant characteristics, and cases of respiratory illness or infection into a standardized form. A second reviewer completed quality control on the extracted data to ensure its integrity. We combined data from fit tested and non‐fit tested N95 respirator groups for MacIntyre 2011 to generate a single comparator N95 group. 16 Similarly, we combined targeted use of N95 respirators with non‐targeted use of N95 respirators for MacIntrye 2013. 17 For Radonovich 2019, we used healthcare worker‐seasons as the population metric (denominator) given that healthcare workers were treated independently and allowed to participate for up to all 4 years the study was conducted (2011/12 to 2014/15). 18

9Reviewers assessed risk of bias of eligible RCTs independently and in duplicate using a modified Cochrane risk of bias tool. 19 Selection bias (random sequence generation and allocation concealment), performance bias (blinding of participants and personnel and other threats to validity), detection bias (blinding of outcome assessment and other potential threats to validity), attrition bias (incomplete outcome data), and reporting bias (selective outcome reporting assessed by comparing outcomes reported in the protocol to those in the published study or by comparing outcomes reported in the results to those in the methods of the published study) were assessed. For each domain in the tool, trials judged as definitely or probably being free of a given risk of bias were considered low risk of bias, whereas trials judged as probably or definitely biased were considered high risk of bias to reduce reporting of unclear bias assessments. For each outcome, we considered individual trials to be at serious risk of bias overall if 2 of the 8 risk of bias domains were judged as high risk and very serious risk of bias overall if more than 2 domains were judged as high risk. Similarly, reviewers assessed the certainty in the evidence independently and in duplicate using the grading of recommendations assessment, development and evaluation (GRADE) approach. 20 The certainty we can have in our evidence ranges from very low, low, and moderate to high. It depends on risk of bias, inconsistency, indirectness, imprecision, and other considerations like publication bias. The Cochrane risk of bias tool and GRADE were applied at the outcome level. Consensus was reached through discussion among the review pair or with consultation of a third reviewer when necessary.

10Pooled odds ratios (ORs) and corresponding 95% confidence intervals (CIs) comparing medical masks to N95 respirators on dichotomous outcomes were calculated in R Project for Statistical Computing (version 3.6.3). The “metafor” package was used, applying the inverse variance method and assuming a random effects model due to expected heterogeneity between studies. 21, 22 We set the criterion for statistical significance at alpha = 0.05. Visual inspection of forest plots and the chi‐square test were performed to evaluate heterogeneity. An I2 statistic value of 0%‐40%, 30%‐60%, 50%‐90%, or 75%‐100% was interpreted as not likely important, moderate, substantial, or considerable heterogeneity, respectively. 23 When inconsistent magnitudes and directions of effect were observed upon visual inspection of the forest plot, and the chi‐square test was significant, we interpreted heterogeneity as more important (ie, interpretation corresponding to the higher range in overlapping I 2 statistic values was reported). 23

11To avoid unit‐of‐analysis errors in pooling data from cluster‐RCTs with individual participant RCTs, we adjusted meta‐analyses by calculating the effective sample sizes of included cluster‐RCTs. We used data on the average cluster size and intraclass correlation coefficient (ICC) to calculate the design effect of the cluster‐RCT when not reported. Individual level data were divided by the design effect to calculate the effective sample sizes (ie, number of events in each trial arm and the total sample size of each trial arm were reduced by the amount of correlation in clusters). We rounded effective sample sizes to the nearest whole number to be meta‐analyzed. 24 Aggregate data from the trials, corresponding effective sample sizes, and the statistical parameters used to calculate the effective sample sizes are available upon request.

12This study was conducted without financial support.

13Our systematic review update identified 1 new randomized trial (n = 5180) eligible for meta‐analysis following screening of 389 titles and abstracts, and 12 full‐texts (Figure 1). 18 To date, there have been four trials where healthcare workers providing care for patients with acute febrile illness were randomized to medical masks (n = 3957) or N95 respirators (n = 4779), of which 3 were identified from a 2016 systematic review (Table 1). 16, 17, 18, 25 Three of the trials were cluster‐randomized, 16, 17, 18 and one was not. 25 Two trials were conducted in North America (Canada and US), 18, 25 and two were conducted in China. 16, 17 All randomized trials reported on laboratory‐confirmed viral respiratory infection, defined by the detection of viral RNA using reverse‐transcriptase PCR from nasopharyngeal and flocked nasal specimens. 16, 17, 18, 25 All studies included PCR testing for respiratory viruses in the Coronavidiae family 16, 17, 18, 25; however, only one trial reported results directly on coronavirus (OC43, 229E, SARS, NL63, and HKU1) infection. 25 Laboratory‐confirmed influenza infection (using PCR or hemagglutination inhibition) and influenza‐like illness (based on pre‐determined respiratory symptoms and fever ≥ 38˚C) were also studied in all 4 randomized trials. 16, 17, 18, 25 The criteria for clinical respiratory illness varied among trials and are detailed in the Appendix S3. 16, 17, 18 Only Loeb 2009 reported on workplace absenteeism. 25

14For laboratory‐confirmed viral respiratory infection, the pooled effect of medical masks compared to N95 respirators was OR 1.06 (95% CI 0.90‐1.25), I 2 = 0% (Figure 2A). For laboratory‐confirmed influenza infection, OR 0.94 (95% CI: 0.73‐1.20), I 2 = 0% (Figure 2B). For influenza‐like illness, the effect was OR 1.31 (95% CI: 0.94‐1.85), I 2 = 5% (Figure 2C). 16, 17, 18, 25 For clinical respiratory illness, OR 1.49 (95% CI: 0.98‐2.28), I 2 = 78% (Figure 2D). 16, 17, 18 Since only one trial reported directly on coronavirus and workplace absenteeism; meta‐analysis was not possible. 25 When seasonal coronavirus (OC43, HKU1, 229E, NL63) was tested for by PCR in this non‐cluster randomized trial of medical masks vs N95 respirators, 4.3% (9/212) of nurses in the medical mask group had RT‐PCR confirmed coronavirus infection compared with 5.7% (12/210) in the N95 respirator group (P =.49). Work‐related absenteeism was reported in 19.8% (42/212) of nurses in the medical mask group compared with 18.6% (39/210) of nurses in the N95 respirator group (P =.75). 25

15Using the Cochrane risk of bias tool, randomized trials were judged to have low risk of selection, attrition and reporting biases. Risk of performance bias was high across all trials and outcomes due to lack of blinding of participant healthcare workers. Risk of detection bias was high for influenza‐like and clinical respiratory illness due to lack of laboratory confirmation, but low for laboratory‐confirmed viral respiratory and influenza infection (Appendix S4). In applying GRADE, the certainty of the overall evidence was judged to be low for laboratory‐confirmed respiratory infection and laboratory‐confirmed influenza infection, and very low for influenza‐like illness and clinical respiratory illness, largely due to indirectness and imprecision (Table 2).

16There is no convincing evidence that medical masks are inferior to N95 respirators for protecting healthcare workers against laboratory‐confirmed viral respiratory infections during routine care and non–aerosol‐generating procedures. Medical masks also performed similarly to N95 respirators in preventing laboratory‐confirmed influenza infection. For influenza‐like illnesses and clinical respiratory illnesses, the point estimates favored N95 respirators; however, the confidence intervals were wide and there was considerable heterogeneity for the clinical respiratory illness outcome (P =.01, I 2 = 78%). This heterogeneity may have been due to the subjective nature of the criteria used to define this outcome between trials (Appendix S3). Reduced protection with medical masks during routine care of COVID‐19 patients cannot be ruled out. Our low certainty in available evidence is because of its indirectness. When we searched for randomized trials comparing the protective effect of medical masks to N95 respirators against coronaviruses, we did not identify any for novel SARS‐CoV‐2 causing COVID‐19.

17Our findings support preliminary epidemiological data from a case‐report of respiratory protective devices for COVID‐19. 26 Forty‐one healthcare workers were exposed to aerosol‐generating procedures from a patient with severe pneumonia, who later tested positive for SARS‐CoV‐2 during COVID‐19 surveillance. These procedures included endotracheal intubation, extubation, non‐invasive ventilation, and exposure to aerosols in an open circuit. All of the exposed healthcare workers tested negative 14 days after their date of exposure, despite 85% (35/41) having worn surgical masks during the high‐risk procedures. 26 Given the limited direct evidence from this case‐report, further research on the risk of secondary infection in healthcare workers caring for COVID‐19 patients is warranted.

18There are several limitations in this meta‐analysis. First, only one trial individually studied cases of coronavirus infection between medical masks and N95 respirators; therefore, we were unable to meta‐analyze coronavirus infection specifically. This led us to downgrade the evidence with GRADE as it relates to indirectness because our findings may not be generalizable to SARS‐CoV‐2. All trials however did report a composite outcome of laboratory‐confirmed viral respiratory infections that included coronavirus infections. Second, this is a meta‐analysis of aggregate data, rather than individual data. The latter would allow for harmonization of confounding co‐variates and outcome definitions, specific to coronavirus infection. The definition of influenza‐like illness was based on a pre‐determined set of signs and symptoms; however, swabs in these studies were likely obtained using a more lenient threshold. Therefore, there would have been swabs obtained even when the influenza‐like illness criteria were not met. This likely resulted in fewer influenza‐like illness events compared with laboratory‐confirmed influenza infections. Reassuringly, outcome definitions (excluding clinical respiratory illness) were consistent among all four studies.

19A strength of this review is that it is up‐to‐date and incorporates the largest randomized trial of medical masks vs N95 respirators that have been completed to date. 18 Secondly, we used appropriate meta‐analytic techniques that accounted for cluster randomization that was present in three of the four included trials. 16, 17, 18 Cluster randomization was not adjusted for in the most recent meta‐analysis potentially leading to falsely narrower confidence intervals around point estimates of protection. 13, 27

20For aerosol‐generating procedures, N95 respirators are unanimously recommended by national and international guidelines; however, there is inconsistency in recommendations for routine care and non–aerosol‐generating procedures of COVID‐19 patients. 6, 7, 8, 9 Our evidence is in keeping with current WHO and Public Health Agency of Canada recommendations to use medical masks for non–aerosol‐generating procedures when caring for COVID‐19 patients. In contrast, the CDC and ECDC recommend use of N95 respirators for non–aerosol‐generating procedures over the less expensive and more readily available medical masks.

21With the widespread of SARS‐CoV‐2, a serious concern is that stockpiles of N95 respirators will be depleted. The Department of Health and Human Services announced that its Strategic National Stockpile—the emergency stockpile of drugs and medical supplies in the United States—contained approximately 30 million medical masks and 12 million N95 respirators. This stockpile of respiratory protective devices equates to 1% of the estimated amount needed for U.S HCWs in a pandemic scenario (42 million stockpiled compared with the estimated 3.5 billion needed). 28 Based on the evidence, preservation of N95 respirators for high‐risk, aerosol‐generating procedures in this pandemic should be considered when in short supply. The uncertainty of this evidence and the depleting stockpiles of respiratory protective devices emphasize the need for further comparative research of medical masks and N95 respirators.

22Jessica J. Bartoszko: Data curation‐Equal, Formal analysis‐Lead, Methodology‐Lead, Project administration‐Lead, Software‐Equal, Visualization‐Equal, Writing‐original draft‐Equal, Writing‐review & editing‐Equal. Mohammed Abdul Malik Farooqi: Data curation‐Equal, Formal analysis‐Supporting, Visualization‐Equal, Writing‐original draft‐Equal, Writing‐review & editing‐Equal. Waleed Alhazzani: Conceptualization‐Equal, Investigation‐Equal, Supervision‐Equal, Writing‐review & editing‐Equal. Mark Loeb: Conceptualization‐Equal, Investigation‐Equal, Supervision‐Equal, Writing‐review & editing‐Lead.