Evaluation of the safety profile of COVID-19 vaccines: a rapid review

1The first coronavirus disease 2019 (COVID-19) case was reported in December 2019. As of June 15, 2021, more than 175 million COVID-19 cases, including over 3.8 million deaths, were reported in 221 countries and territories. In response to the COVID-19 pandemic, 102 candidate vaccines on 10 platforms are in clinical development, and 15 vaccines have already been licensed or approved for emergency use.

2These platforms can be classified either as traditional approaches that have previously resulted in licensed vaccines (e.g., inactivated, recombinant proteins, vectored vaccines), or as approaches that have never before been used for a licensed vaccine (e.g., RNA and DNA vaccines). Since no vaccine against coronaviruses had ever been licensed for use in humans before, the rapid process of research and development and limited follow-up time post-vaccination aroused great public concern about the safety profile of COVID-19 vaccine candidates, especially for new platforms such as RNA vaccines. Common reasons given for not intending to receive these vaccines included “concern about the safety of the vaccine in its development” and “potential side effects”. As mass vaccination has progressed, more occurrences of adverse events following immunization (AEFI) have been reported, especially the rare AEFIs. Demonstrating and summarizing vaccine safety from clinical trials and post-authorization surveillance is critical for public confidence, and for enabling timely, evidence-based policy decisions for population-level use.

3Current evidence about the safety of COVID-19 vaccines relies mainly on data from phase 1–3 randomized controlled trials and vaccine safety surveillance system in several countries. We found three reviews of the safety of COVID-19 vaccines, which combined study experimental groups, and did not examine the heterogeneity between vaccine platforms and participant age groups. Here, we conduct a rapid review and meta-analysis to summarize the safety data of COVID-19 vaccine candidates. We aim to comprehensively evaluate the rate of solicited, unsolicited, and serious AEFI in each clinical trial and to estimate the relative risk of AEFI by vaccine platform and participant age group. We also collected post-authorization surveillance data from around the world to look for uncommon and delayed onset reactions. This overview of the safety profile of COVID-19 vaccines will support responses to potential safety issues and inform decision-makers evaluating vaccination strategies around the globe.

4We conducted a rapid review, adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses standards whenever possible. For the published results of clinical trials, we searched PubMed, Embase, and Web of Science for peer-reviewed articles, and PMC for preprints. We also used various combinations of the search terms “severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)”, “coronavirus”, “vaccines”, “safety”, “adverse event”, and “side effect” to identify relevant regulatory documents disclosing experimental and surveillance data. In addition, we searched official websites and reports using terms for “COVID-19 vaccine safety monitor/monitoring/surveillance” and the names of countries with COVID-19 vaccine programs to identify their available safety surveillance data. Searches were conducted as of June 12, 2021. Details of the search strategy are presented in Additional file 1: Table S1.

5Three researchers (Q.W., X.C., X.B.) assessed eligible studies, conducted data extraction, and cross-checked. We looked for clinical trials and post-authorization reports that examined safety data of COVID-19 candidate vaccines, and included manuscripts published in peer-reviewed journals, preprints, and unpublished data disclosed by regulatory agencies. No restrictions were placed on publication date. We excluded study protocols, media news, commentaries, reviews, case reports, reports of non-human clinical trials, reports among specific populations (such as pregnant and lactating women, cancer patients, and other immunosuppressed persons), and abstracts of congress meetings or conference proceedings. We also excluded interim reports of clinical trials that did not clearly show safety data of specific COVID-19 candidate vaccines selected for further use, and reports on vaccines no longer under clinical evaluation. Post-authorization observational studies with sample sizes less than one thousand were excluded as well.

6Information extracted from qualified studies included basic clinical trial details (e.g., study design, study location, phase, arms), characteristics of subjects (e.g., age group, proportion of subjects with underlying conditions, proportion of subjects seropositive at baseline), vaccine formulations (e.g., antigen content, adjuvant, injection route, vaccination schedule), the number of subjects in the safety dataset, and the rate of AEFI during the follow-up period. If data for the same subjects were presented in multiple publications, these data were only counted once. Due to phase 1 and 2 trials often including multiple differing experimental groups, we focused exclusively on safety data from experimental groups in phase 3 clinical trials. Any discrepancies were resolved by consensus or in consultation with a third researcher. Two researchers (Q.W., X.C.) assessed the methodological quality of studies using the Cochrane risk of bias tool. Disagreements were resolved by consensus. Certainty of evidence was assessed by researchers according to the grading recommendations assessment, development and evaluation (GRADE) framework.

7For the safety profile of COVID-19 vaccines in clinical trials, the primary outcomes were the proportion of vaccine recipients experiencing at least one AEFI and the rates of selected AEFI of COVID-19 candidate vaccines versus placebos. We specified severe versus mild-to-moderate AEFI in our extraction and analyzed these categories separately. For post-authorization safety data, we examined rates of AEFIs, serious adverse events (SAE), and adverse events of special interest (AESI).

8We performed meta-analyses of proportions to estimate the pooled rate of safety outcomes of eligible COVID-19 vaccines (i.e., those both with reports of phase 1–3 trials and still under ongoing clinical evaluation) using different platforms. In addition, we estimated the pooled rate ratio (RR) using the rate of safety outcomes of COVID-19 vaccines in vaccinated groups divided by those in control groups in each study. We synthesized evidence for the following events: local reactions (e.g., injection-site pain, injection-site induration, tenderness, swelling), systemic reactions (e.g., nausea, vomiting, fever, rash, myalgia, arthralgia, headache, fatigue, malaise, diarrhea, cough), unsolicited AEFI by system organ class (SOC), AESI, serious AEFI, medically attended events, and study withdrawal of subjects as a result of AEFI and death. Definitions of the study outcomes and the grading scale of selected AEFI were provided in Additional file 1: Table S2-S3.

9We explored the reasons for variations among eligible vaccines and examined whether rate of AEFI varied by vaccine platform, age group of participants and serostatus of participants against SARS-CoV-2 at baseline. For the purposes of stratifying safety data by age group, we defined younger adults as under 65 years of age and elderly as over 65 years of age. If the age group of the clinical trial was not completely consistent with our study, the safety data of the closest age stratification was extracted. We classified all participants in the Ad5 nCoV trials as younger adults, since no stratified analyses by age were performed and the proportion of the participants under 55 years old was reported as 86%.

10Based on a random-effect meta-analysis model, we used the inverse variance method to estimate pooled rate by platform, and the Clopper-Pearson method to calculate 95% confidence intervals. Heterogeneity tests (chi-squared test) with Higgins’ I2 statistics were used to determine the extent of variation between vaccines. Multivariate meta-regression models were used to determine the relative contribution of vaccine platform and age of participants to the rate of AEFI. All meta-analyses were performed using per-protocol data. Small study effects (potentially caused by publication bias) were assessed using funnel plots, and formally tested through the rank correlation test for those meta-analyses including more than 10 studies. All statistical analyses were done using R (version 4.0.2), using the “meta” package to conduct the meta-analysis. For all statistical tests, two tailed P-value less than 0.05 were considered statistically significant.

11Our search identified a total of 7231 records after removal of duplicates (Fig. 1). After initial title/abstract screening, 157 articles were assessed for eligibility via full-text review. For the safety data among general population, 43 articles reporting on 19 vaccines of 6 different platforms and 10 documents released by WHO (World Health Organization), US Food and Drug Administration (FDA) and UK Medicines & Healthcare products Regulatory Agency (MHRA) from clinical trials were included. A total of 123,540 study participants receiving COVID-19 vaccines and 97,944 participants receiving placebos were included in safety set of clinical trials. Post-authorization safety profiles were assessed through 3 reports released by the European Medicines Agency (EMA), 20 reports including large-scale monitoring data, 11 observational studies, and 26 reports from countries’ national surveillance systems.

12The main characteristics of included vaccines and relevant clinical trials are reported in Table 1 and Additional file 1: Table S4. The methodological quality of the included studies is reported in Additional file 1: Table S5-S6. Interim and/or final reports of phase 3 clinical trials were available for 8 vaccines: BNT162b2 (RNA vaccine manufactured by Pfizer and BioNTech), mRNA-1273 (RNA vaccine manufactured by Moderna), ChAdOx1-nCoV (non-replicating vector vaccine manufactured by Oxford and AstraZeneca), Gam-COVID-Vac (non-replicating vector vaccine manufactured by Gamaleya Research Institute), Ad26.COV2.S (non-replicating vector vaccine manufactured by Janssen Vaccines & Prevention B.V.), CoronaVac (inactivated vaccine manufactured by SinoVac), BBIBP-CorV and WBIP (inactivated vaccine manufactured by Sinopharm) (Table 1). AEFIs were mainly graded according to the latest scales issued by the US FDA and the China State Food and Drug Administration (CFDA), which are very similar except for a difference of 0.3–0.5 °C in the definition of fever (Table 1 and Additional file 1: Table S3). The funnel plots for safety outcomes including local reaction, systemic reaction, and medically attended events did not appear to be skewed, and the corresponding rank correlation test did not identify asymmetry (Additional file 1: Figure S1).

13The pooled rates of local and systemic reactions, respectively, were significantly lower among inactivated vaccines (23.7%, 21.0%), protein subunit vaccine (33.0%, 22.3%), and DNA vaccines (39.5%, 29.3%) than the 3 other types of COVID-19 vaccines (RNA vaccines, 89.4%, 83.3%; non-replicating vector vaccines, 55.9%, 66.3%; virus-like particle vaccines, 100%, 78.9%) (Figs. 2 and 3). Among all vaccines, solicited injection-site pain and tenderness were the most common local reactions, and fatigue and headache were the most common systemic reactions (Additional file 1: Table S7). Compared to controls, the highest risk of local reactions (RR 4.5, 95% Cl 3.4–5.9) was observed for protein subunit vaccines (Table 2), and a higher risk of medically attended events (RR 1.7, 95% Cl 1.3–2.2) was observed for RNA vaccines (Table 2).

14For RNA and non-replicating vector vaccines, most unsolicited AEFI and highest risk of unsolicited AEFI by SOC within 28 days post-vaccination were general disorders and administration site conditions, and the rate of common AEFI by SOC was significantly different among vaccines (Additional file 1: Figure S2-S3). The most common serious AEFI by SOC was infections and infestations, while the rate of identified serious AEFIs was similar in the overall vaccine and placebo groups (Additional file 1: Table S8).

15For vaccine-related serious AEFI, there was no difference between vaccine and placebo groups (Table 2 and Additional file 1: Table S9). For adverse events of special interest (AESI), approximately 1% and 0.6% of participants vaccinated with RNA vaccines reported hypersensitivity and lymphadenopathy, respectively, and potential risk of hypersensitivity and lymphadenopathy was observed in RNA vaccines compared to control groups (Additional file 1: Table S10). It was worth noting that a total of 7 cases of Bell’s palsy were identified among 36,805 RNA vaccine recipients, indicating a numerical imbalance compared to placebo (Additional file 1: Table S10). There was no imbalance in the number of reported SAEs or grade 3 and over adverse events between vaccine and placebo groups for CoronaVac, BBIBP-CorV, and WBIP.

16The rate of the most common solicited symptoms was significantly higher among younger adults compared to the elderly (Additional file 1: Table S11). RNA vaccines had significantly higher rate of most common solicited reactions (e.g., injection-site pain, fatigue, headache) among younger adults compared to the other 5 platforms, regardless of the grades of adverse reactions (except overall injection-site pain which was also quite high for virus-like particle vaccines) (Additional file 1: Figures S4-S6). Meanwhile, the highest risk of these common systemic reactions (including fever) was observed in RNA vaccine recipients in this age group, compared to controls (Additional file 1: Table S12). While the highest rate of fever was shown in virus-like particle vaccines (Additional file 1: Figure S7). Differences between vaccine platforms and age groups of vaccine recipients accounted for much of the heterogeneity in safety profiles between COVID-19 vaccines (Additional file 1: Table S13). In addition, the rate of AEFI after CoronaVac was less frequent in children and adolescents than in younger adults, whereas the reverse was found with BNT162b2 (Additional file 1: Table S7).

17The most common AEFIs observed in post-authorization observational studies were local injection pain, fatigue, and headache (Additional file 1: Table S14). Adverse events were more frequent in females and subjects with a history of SARS-CoV-2 infection, and decreased with age (Additional file 1: Table S14). Several studies explored COVID-19 vaccination safety signals, including anaphylaxis, cerebral venous sinus thrombosis (CVST), thrombocytopenia, myocarditis, and pericarditis.

18Nationwide safety surveillance data for COVID-19 vaccines (mainly BNT162b2, mRNA-1273, ChAdOx1, and nCoV-19) were reported in 26 countries (Additional file 1: Table S15). Most of this reporting was based on passive surveillance and thus not necessarily indicative of true rates or causal relationships with vaccination. Crude reporting rates of common AEFI and SAE varied between countries and were lower than that in clinical trials (Table 3, Additional file 1: Table S16). National rates of anaphylaxis ranged from 2.5 to 15.8 per million doses after mRNA COVID-19 vaccination and were estimated at 0.8 per million doses after Sinopharm vaccination and < 0.5 per million doses after Janssen vaccination (Additional file 1: Table S16).

19The pooled rates of local and systemic reactions were significantly different between vaccine platforms. Inactivated vaccines, protein subunit vaccines, and DNA vaccines had lower rates of local and systemic reactions compared to RNA vaccines, non-replicating vector vaccines, and virus-like particle vaccines. The safety profiles of BNT162b2, mRNA-1273, ChAdOx1-nCoV, Ad26.COV2.S, and CoronaVac were relatively benign in the elderly, and both the frequency and the intensity of local and systemic reactions decreased with age. The rates of SAE, including non-fatal serious AEFI and death, were similar in vaccine and placebo groups in clinical trials. Reporting rates of common AEFI after mass public vaccination were lower than in clinical trials. Several unexpected rare adverse events, which resulted in severe outcomes, have been noted in post-authorization surveillance.

20Differences in safety profiles of vaccines must be considered in the context of efficacy. Both RNA vaccines (BNT162b2 and mRNA-1273) reported 95% and 94% vaccine efficacy, respectively (symptomatic PCR-confirmed cases were the primary clinical trial outcomes). This is substantially higher than the reported efficacy of other vaccine platforms. The efficacy of inactivated vaccines was reported as 78.1% for BBIBP-CorV and 50.7% for CoronaVac. Efficacy of Ad26.COV2.S against moderate to severe critical Covid-19 with onset at least 14 days after administration was 66.9%. Overall efficacy of ChAdOx1-nCoV in preventing symptomatic COVID-19 across both the low dose and standard dose groups was reported as 70.4%. The efficacy of Gam-COVID-Vac, another non-replicating vector vaccine, was 91.6%. Based on the current evidence, RNA vaccines have both higher rates of adverse reactions and higher efficacy. Due to the relative mild and transient nature of most of these reactions, RNA vaccines should be considered an excellent option to protect against COVID-19, especially in the absence of other viable candidates with similar efficacy. In addition to safety and efficacy, vaccine candidates must also be assessed in the context of the risk of disease, to determine whether each vaccine supports a favorable benefit-risk ratio or not. Such a determination is undoubtedly more important than comparing safety and efficacy between vaccine candidates as long as vaccine supply is limited and disease is prevalent.

21Direct comparisons between efficacy data should also be interpreted with caution due to the inconsistency of environmental risk, endpoints, and statistical methods between studies. Current efficacy data show that all authorized vaccines exceed the 50% threshold set by WHO, indicating they prevent substantial disease, especially severe cases. Authorized COVID-19 vaccines can prevent a large proportion of symptomatic cases, hospitalizations, severe diseases, and death. Mass vaccination efforts can prevent disease, save lives, reduce pressure on the medical system, and hopefully eventually relieve the need for many of the non-pharmaceutical interventions currently used to contain the epidemic, reopen economies, and allow a return to normalcy worldwide.

22As of May 9, 2021, about 0.6 billion people around the world had been vaccinated with at least one dose of COVID-19 vaccines, accounting for about 7.8% of the world’s population. This mass vaccination should allow for the identification of more uncommon and rare AEFI. According to the Vaccine Adverse Event Reporting System (VAERS) and V-safe system of the US Centers for Disease Control and Prevention (CDC), the rates of non-serious AEFI after public administration of BNT162b2 and mRNA-1273 were similar to the clinical trials. Anaphylaxis, a severe, life-threatening allergic reaction, typically occurs at a rate of approximately 1 case per million doses for most vaccines; the rates of anaphylaxis associated with BNT162b2 and mRNA-1273 appear to be approximately 4.7 times and 2.5 times higher than this, respectively, although no cases progressed to serious long-term outcomes thanks to their prompt treatment. Variations in the incidence of anaphylaxis between countries are to be expected, as the numbers vaccinated in most countries to date are relatively small compared with the USA, and the reporting rates of AEFI from passive surveillance are biased. A causal link of thrombosis and thrombocytopenia with adenoviral vector vaccines (ChAdOx1 nCoV-19 and Ad26.COV2.S) was noted after mass public vaccination, including several deaths and severe outcomes. While rare side effects should not derail vaccination efforts, a thorough risk-benefit analysis is required. Several studies have explored the safety profile of two mRNA vaccines (BNT162b2 and mRNA-1273) in HIV-positive populations, immunosuppressive patients, and pregnant women, revealing no evidence of unexpected serious adverse events. Further evaluation of the benefit-risk profile is warranted in these specific populations.

23According to the Chinese government, 333 million doses have been administrated as of May 10, 2021 (mainly with BBIBP-CorV and CoronaVac), and the rate of overall AEFI was close to the previous inactivated vaccines given routinely, while the rate of allergic reactions and other non-fatal serious AEFI was about 2 cases per million doses. No major safety concerns have been identified so far. Safety data on Russian vaccines need to be disclosed further so that safety signals can be identified and appropriate risk minimization measures quickly implemented.

24The safety profiles of COVID-19 vaccines are still incomplete, even for those currently in use. The safety and efficacy of COVID-19 vaccines in certain subpopulations, such as children and adolescents, pregnant woman, and people with multiple underlying conditions, have not yet been fully studied. Although crude reporting rates of AEFIs from post-authorization safety monitoring have so far been lower than in clinical trials, adverse reactions that are uncommon or have delayed onset require extended post-authorization study to detect. Investigation of safety signals, a lack of epidemiological tools for active surveillance, obstacles at the national regulatory authority level, and a lack of information sharing between countries are still major challenges for most countries. Pharmacovigilance mechanisms must be put in place, with all the necessary training, especially in low- and middle-income countries. Further study will strengthen and expand upon our knowledge in these areas and enable the refinement of vaccine recommendations and injury compensation programs. Safety issues noted in mass vaccination may have a deleterious impact on the global vaccine supply and the already fragile confidence in vaccines. The benefits of vaccines still outweigh the risks at present. Government agencies and vaccine developers should continue to take action to encourage vaccination and reduce public vaccine hesitancy.

25Our analysis has several limitations. Firstly, we only included data reported at the study level, due to limited access to individual-level data. Secondly, there are factors we did not include in the meta-analysis, such as seropositivity against SARS-CoV-2 at baseline and underlying conditions, so the potential effects of such heterogeneity were not quantitatively assessed. Thirdly, in the clinical trials for BNT162b2 and ChAdOx1-nCoV, age groups were divided at 55 years of age, which differed from our subgroup analysis of age divided at 65 years of age. Finally, although we included currently available post-authorization safety monitoring data, such monitoring programs are still in their infancy and often rely on a mix of active and passive surveillance.

26In conclusion, the available evidence indicates that eligible COVID-19 vaccines have an acceptable short-term safety profile. Additional studies and long-term population-level surveillance are strongly encouraged to further augment the safety profile of COVID-19 vaccines. This should include essential active vaccine safety surveillance systems, enhanced monitoring of early COVID-19 vaccine recipients and passive surveillance, standardized reporting and pharmacovigilance mechanisms, platforms in hospitals to evaluate the vaccine-specific antibody correlates, and cross-reactivity to other strains. All reports of suspected adverse reactions should be investigated and warning signals rapidly evaluated, to allow implementation of appropriate risk minimization measures and update the benefit/risk ratio of vaccination.