Comparative systematic review and meta-analysis of reactogenicity, immunogenicity and efficacy of vaccines against SARS-CoV-2

1Three novel coronaviruses (HCoV) have crossed into humans during the 21st century. Severe acute respiratory syndrome 1 coronavirus (SARS-CoV-1) emerged in China in 2002/3, with 8096 infections and about 10% case fatalities. In 2012, a SARS-like disease emerged in Saudi Arabia, termed Middle East respiratory syndrome coronavirus (MERS-CoV), sporadic outbreaks leading to 2519 infections and a case fatality rate of 35%. The knowledge accrued in relation to protective immunity and vaccinology of SARS-CoV-1 and MERS-CoV is pertinent to decoding the principles of protection from the highly related SARS-CoV-2.

2Various vaccine candidates were developed following the emergence of SARS-CoV-1 and MERS-CoV, with live-attenuated, DNA, and recombinant viral vectors investigated–. With an animal model in place, advances in vector design and greater knowledge of disease pathogenesis, there was significant pre-clinical and clinical research towards an effective MERS-CoV vaccine. One MERS-CoV vaccine candidate lowered viral shedding in the dromedary camel reservoir. Development of SARS-CoV-1 vaccines began during the outbreak, but subsided as the threat of a major pandemic decreased.

3With respect to the SARS-CoV-2 pandemic, there has been unprecedently rapid vaccine development: the Pfizer-BioNTech and Moderna mRNA vaccines and the Astra Zeneca/Oxford ChadOx vaccines are widely rolled-out in several countries as well as some countries utilizing the Chinese Sinovac inactivated SAR-CoV-2 and Russian Sputnik V adenovirus vaccines, and over 300 candidates at different stages of development, utilizing diverse platforms, including protein subunit with adjuvant, non-replicating viral vectors, RNA, virus-like-particles (VLP), DNA, inactivated- and live-attenuated virus,. An ideal vaccine is one that can be produced at scale and low cost, is safe, easy to distribute and store, induces strong, protective neutralizing antibody and T cell responses, ideally with a single dose, elicits a durable response that does not recapitulate the waning antibody (Ab) titers seen following natural coronavirus infection, and should be equally suitable for very young, old and immunosuppressed individuals. It should also be technically modifiable to accommodate improving efficacy against emerging variants. To achieve these deliverables in the context of the COVID-19 pandemic, several vaccines may be required. Following an initial period awaiting trial results of vaccine candidates, we have entered a period now in which the public, policy-makers and researchers start to consider detailed, comparative questions raised by the extraordinary, real-life challenges of mass vaccination roll-out during an ongoing, global, pandemic. This has focused greater attention on dosing and boost intervals and the quality, quantity and durability of immune responses, as well as details of the adverse event (AE) profile. Past coronavirus vaccine research has highlighted potential safety concerns related to antibody (Ab)-dependant enhancement (ADE) and induction of Th2-associated lung immunopathology following viral challenge,.

4At a time of intense vaccine development across many diverse platforms, there is an incentive to address the need for a comprehensive appraisal benefiting from experience from coronavirus vaccine trials. The investigated approaches encompass different advantages and disadvantages with respect to immunogenicity, efficacy, durability of immunity, safety profile and ease of manufacture, yet there has thus far been relatively little side-by-side evaluation. The aim here was to evaluate pertinent SARS-CoV-1, MERS-CoV and SARS-CoV-2 studies in humans and NHP published up to the March 22, 2021. In the context of the current pandemic this is pertinent to structuring rational, comparative appraisal and selection of the most practical and effective SARS-CoV-2 vaccines.

5Study selection followed PRISMA guidelines (Fig. 1). A search of PubMed and EMBASE databases used pre-determined search criteria, initially generating 5945 studies (Supplementary Tables 1 and 2). Duplicates, in vitro studies and studies using other animal models were not taken forward. Searches by ‘Title and Abstract’ yielded a total of 55 articles updated to March 22, 2021. Thirty-two human– and 23 NHP– studies about vaccines against SARS-CoV-1, MERS-CoV and SARS-CoV-2 are reviewed (Supplementary Tables 3 and 4). Individual vaccines are summarized in Table 1 and Supplementary Table 5. The greatest focus of research activity was in China and the USA (Supplementary Fig. 1). The mean age in human trials was 42.7 years. The gender distribution in human vaccine groups was roughly equal. Vaccine group ethnicity was skewed as follows: 72.8% of participants were White, 11.2% Hispanic, 7.2% Black/African American and 3.9% Asian (Supplementary Table 6).

6There were many different vaccine platforms used, the most frequent being viral vector platforms, followed by nucleic acid (mRNA/DNA) and inactivated virus platforms. Excluding the inactivated vaccine platforms, the predominant antigen insert employed was a variant of, or the full length, spike protein—the defined nAb target. The vast majority of vaccine schedules use a 2-dose prime/boost protocol, with the interval between doses ranging from 14 to 84 days and boost at day 21 being the most commonly reported protocol (Table 1 and Supplementary Tables 3–5).

7Antibody binding studies were largely carried out by ELISA-IgG in NHP and human vaccine studies. One NHP study used ELISA-IgA to evaluate mucosal immunity. Antigens used for IgG detection in humans trials were spike GP, spike RBD and trimeric spike GP. T cell analysis was primarily by IFNγ ELISpot or by intracellular cytokine staining (ICS), generally using overlapping peptide pools for spike antigen. We have not attempted a cross-comparison between these two readouts to arrive at a universal comparison of spike-specific responder cell frequency since differential strategies for CD4/CD8 gating in ICS impose a confounder.

8Of the five published human studies evaluating vaccine efficacy, the Polack, BNT162b2 (Pfizer-BioNTech) and Baden, mRNA-1273 (Moderna vaccine) vaccines showed the highest efficacy at greater than 94.6% followed by the Logunov, rAd26/rAd5 vaccine at 91.6% efficacy. The Voysey,, ChAdOx (Astra Zeneca/Oxford Vaccine) demonstrated efficacy of 70.4% and 66.7% in two pooled efficacy studies (Supplementary Table 7). For efficacy and safety analysis of vaccine candidates tested in NHP studies, histopathology, biochemical analysis, clinical and radiological evaluations were most often used, along with RT-PCR measure of viral load (Supplementary Tables 4 and 8).

9NHP safety analysis was described in seven studies (Supplementary Table 8). There were few notable changes in biochemical, hematological or clinical signs in the majority of NHP vaccine groups. Four studies evaluated ADE risk,,,, suggested to be a risk factor in HCoV vaccine safety, and none of the studies found evidence of ADE. No significant Th2 pathology was found in any NHP analysis.

10Safety analysis of local and systemic AE were recorded across all 32 human studies. Figure 2 shows a comparative Forest plot analysis of the incidence of AE across published human studies looking at the relative risk in vaccine compared to control groups. Overall, there is a 1.7-fold increased risk of any AE in the vaccinated compared to control groups. There was a 4.1-fold increased risk in Polack, BNT162b2 mRNA vaccine and a 1.8-fold increased risk in the Baden, mRNA-1273 vaccine (Fig. 2). The vast majority of reported AE were in the mild local and systemic category.

11In terms of mild or moderate local AE, the most commonly reported local AE was injection site pain, followed by redness and swelling (Fig. 3 and Supplementary Table 9). Martin DNA-SARS-CoV-1, Modjarrad DNA-MERS-CoV, Baden/Jackson/Anderson/Chu mRNA-1273-SARS-CoV-2–, Walsh/Mulligan/Pollack BNT162b1/2-SARS-CoV-2,,, Logunov rAd26/rAd5-SARS-COV-2, and Folegatti/Ramasamy, SARS-CoV-2 ChAdOx, and Keech NVX-CoV2373 vaccines all reported a high percentage of subjects with injection site pain. Koch MERS-CoV and Martin DNA-SARS-CoV-1 reported the highest incidence of local redness and swelling.

12In terms of mild or moderate systemic AEs, the most commonly reported were headache, fever, myalgia, chills and fatigue (Fig. 4 and Supplementary Table 10). They were reported for most vaccine platforms, including BNT162b1/2-SARS-CoV-2,,, mRNA-1273-SARS-CoV-2–, SARS-CoV-2 ChAdOx,, NVX-CoV2373 and rAd26/rAd5-SARS-COV-2 (Sputnik V), vaccines. There is increased frequency of mild/moderate systemic AE events with increased dose and at the boost of a prime boost schedule seen in the RNA vaccines Baden/Jackson/Anderson mRNA-1273-SARS-CoV-2,,, and Walsh/Mulligan/Pollack BNT162b1/2-SARS-CoV-2,,, and Keech, NVX1273-Cov vaccines with increased incidence of headache, myalgia, arthralgia and fatigue after boosting. Analysis of individual local and systemic AE is available in Figs. 3 and 4 and Supplementary Tables 9 and 10.

13Several of the clinical trials included in this systematic review recorded grade three AE related to vaccine or placebo (Supplementary Fig. 2). Baden, mRNA-1273 showed the highest percentage of grade 3 AE (11%) following boost. Zhu,, AdH5, Keech, NVX-CoV2373, Sadoff, Ad26.CoV2.S, Mulligan/Walsh, BNT162b1/2 and Folegatti, ChAdOx all reported grade 3 AE in more than 3.5% of participants.

14The occurrence of reported solicited and unsolicited serious AE following SARS-CoV-2 vaccination in the published studies to date was low with similar reporting in vaccine and control groups at 0.6% in each (Baden, mRNA-1273); 0.6% vaccine and 0.5% control (Pollack, BNT162b2); 0.7% vaccine and 0.8% control (Voysey, ChAdOx) and 0.3% vaccine and 0.4% control (Logunov, rAd26/rAd5) (Supplementary Table 11).

15Autonomic nervous imbalance, nausea, vomiting, rheumatoid arthritis, dyspnea, swollen face and peripheral edema were reported in the vaccine group of Baden, mRNA-1273. Transverse myelitis and fever were recorded in the experimental arm of Voysey, ChAdOx-SARS-CoV-2. Shoulder injury related to vaccination, right axillary lymphadenopathy, paroxysmal ventricular arrhythmia and right leg paresthesia were recorded in the Polack, BNT162b2-SARS-CoV-2.

16Interstudy analysis of ELISA-IgG GMT was conducted across published human studies–,–,–,,,–,. A number of conclusions can be drawn regarding overall titer and the dose-dependent and boost dependent nature of ELISA-IgG Ab responses (Fig. 5).

17The vaccine achieving the highest Ab titer was the Jackson, Anderson mRNA-1273-SARS-CoV-2, vaccine. It exhibited dose and boosting dependent increases in Ab titer. The high-dose Keech NVX-CoV2373 vaccine elicited similar Ab titers to the unboosted mRNA-1273-SARS-CoV-2 vaccine. The next highest antibody binding titers were seen in the Sahin, Walsh BNT162b1-SARS-CoV-2, and Logunov, rAd26/rAd5-SARS-COV-2 followed by Ella, BBV152,, Sadoff, Ad26 and Folegatti/Ramasamy, SARS-CoV-2 ChAdOx,. Several vaccines display boost-dependant increases in Ab titer including Logunov AdH26-SARS-CoV-2; Ramasamy, Folegatti ChAdOx-SARS-CoV-2,; and Sahin, Walsh BNT162b1-SARS-CoV-2,. Modjarrad DNA-MERS-CoV elicited similar Ab responses to the Zhu, AdH-5-SARS-CoV-2, and Folegatti, ChAdOx-MERS-CoV vaccines, and exhibited dose-dependent increase in responses in the middle and high dose vaccine group. The inactivated virus vaccines included in this review achieved similar Ab responses with minimal boost or dose-dependent increases in titers,,.

18Comparing human and NHP GMT ELISA-IgG responses, human studies reported mean Ab titers similar to ELISA-IgG responses seen in NHP studies (Supplementary Fig. 3A).

19Next we looked at human vaccine trial data for nAb responses (Fig. 6). The adjuvanted Keech, NVX-CoV2373-SARS-CoV-2 and Chu, mRNA-1273 Prime-Boost vaccine groups achieved the highest nAb responses followed by the Sadoff, Ad26-SARS-CoV2 vaccine then Folegatti/Ramasamy, SARS-CoV-2 ChAdOx, and Sahin, Walsh BNT162b1-SARS-CoV-2,. Both are dependent on the boost to achieve high nAb titers. At the vaccination dose of 30 µg, the 56–85 yo cohort of the Walsh, BNT162B1/2 study achieved lower nAb titers than the 15–55 yo cohorts. A similar reduction in nAb titers was seen with increasing age across the cohorts (18–55 yo, 56–70 yo and >70 yo) in Ramasamy, SARS-CoV-2 ChAdOx. This emphasizes the importance of the boosting dose to achieve optimal nAb titers particularly in older age groups. The importance of a boost dose to achieve optimal nAb titers is also seen in the Jackson/Anderson mRNA-1273, and Folegatti, SARS-CoV-2 ChAdOx studies using a pseudotyped virus nAb assay (Fig. 7). Data from the >70 yo age group in the Anderson, mRNA-1273 study (Fig. 7) is noteworthy as boosting is needed to achieve a measurable titer. In Chu, mRNA-1273 there was no reduction in nAb response with increasing age.

20Studies looking at human and NHP live and pseudovirus nAb responses to vaccination report responses of a similar order of magnitude (Supplementary Fig. 3B, C).

21A number of vaccines display convergence between ELISA, pseudotype virus neutralization and live virus nAb titers. In the Anderson mRNA-1273-SARS-CoV vaccine groups there were significant dose-dependent increases in pseudotype virus nAb titer, similar to the ELISA-IgG responses (Figs. 5, 6, and 7). A similar relationship exists between the live nAb and ELISA-IgG responses in the Sahin, BNT162b1, Logunov, AdH26/5-Vac/Lyo and Ramasamy, ChAdOx SARS-CoV-2 vaccines.

22Studies have reported pre-existing Ab against the vaccine viral vector itself. Supplementary Table 12 details pre-existing titers: the Zhu, AdH-5-SARS-CoV-2, vectored vaccines showed high levels of pre-existing nAb with an average of 52.2% (range: 44–58%) of participants having pre-existing nAb (>1:200, titer) to the AdH-5 virus vector. Pre-existing Ab roughly halved the Ab response to the vaccine. About 9.2% (range: 5–13%) of participants in the Logunov, AdH26/5-Vac/Lyo trial had pre-existing anti-vector nAb. However, no significant correlation was found between titers of anti-vector nAb and titers of RBD specific IgG. Only one percent of participants in the Folegatti, ChAdOx-SARS-CoV-2 had pre-existing anti-vector nAb of >1:200 although 19% did have nAb. In the larger Ramasamy, ChAdOx-SARS-CoV-2 trial, there were no pre-existing nAbs, however the anti-ChAdOx nAb titer increased following prime vaccination, but did not after boost vaccination (Supplementary Table 12).

23T cell responses were evaluated using T cell ELISpot IFNγ responses and/or ICS. Ten human studies evaluated T cell ELISpot IFNγ responses (Fig. 8). The highest IFNγ ELISpot responses were seen in the Folegatti, ChAdOx-MERS-CoV and the Ramasamy, ChAdOx-SARS-CoV-2 vaccine studies. The Zhang, CoronaVac-SARS-CoV-2 vaccine study showed somewhat lower IFNγ SFC. Limited analysis of mean IFNγ ELISpot responses of other trials demonstrated dose-dependent increases in responses in all groups of the Modjarrad, DNA-MERS-CoV and Koch, MVA-MERS-CoV vaccines. Overall the log-transformed mean IFNγ ELISpot cell response in the human studies was 2.3 with a range of 2.3. Similar responses were obtained in NHP studies (Supplementary Fig. 3D). As had been seen for the nAb response, the IFNγ ELISpot response to a single 60 µg dose of the Sahin BNT162b1 vaccine was below the lower limit of detection in the majority of vaccinees at d 29.

24ICS used to characterize T cell cytokine responses and subsets was described in 14 human studies (Supplementary Table 13). While it is hard to compare effector cell frequencies directly between ELISpot and ICS platforms (since the latter encompass diverse strategies for co-stimulation, peptide panels, gating and cytokine readouts), it can nevertheless be seen that very high responder frequencies are achieved, often around 0.2% of gated CD4+ cells being cytokine-positive on stimulation; see for example Jackson mRNA-1273-SARS-CoV-2 intermediate dose, or the Keech NVX-CoV2373 Phase 2 trial data. Cytokine-positive CD4+ T cell responses were more prominent than CD8+ T cell responses in most studies and IFNγ+ expression was the most common T cell effector function recorded in human studies. There was little evidence for Th2 cytokine profiles with Th1 bias most prevalent, although some subjects in Keech, NVX-CoV2373 make a detectable IL-5/IL-13 response.

25Five NHP studies evaluated T cell subset responses by ICS. CD4+ and CD8+ subset responses were stimulated in all studies (Supplementary Table 13). CD8+ T cells were the most prominent subset stimulated in NHP trials, in contrast to the findings in human studies. The Yu, DNA-SARS-CoV-2 vaccine elicited minimal CD4/8+ IL-4 responses relative to the CD8/4+IFNγ responses, indicative of a Th1/Tc1 bias; other NHP studies also recorded Th1/Tc1 responses following vaccination,. As in the human data, IFNγ+ effector function was most prominent in NHP studies (Supplementary Table 13). No significant Th2 responses were recorded following vaccination in any NHP study. The Muthumani, DNA-MERS-CoV and Corbett, mRNA-1273-SARS-CoV NHP vaccine groups exhibited dose-dependent increase in T cell cytokine responses.

26Five SARS-CoV-2 human Phase 3 trials reported vaccine efficacy,,,, (Supplementary Table 7). Polack, BNT162b2, Baden, mRNA-1273 and Logunov, rAd26r/Ad5 achieved remarkably high efficacy, 94.6%, 94.1% and 91.6%, respectively. Using pooled data from the UK and Brazil Voysey, ChAdOx reported an efficacy of 70.4%. Follow-up analysis of pooled data from the UK, South Africa and Brazil, Voysey, ChAdOx reported an efficacy of 66.7% (Supplementary Table 7).

27Of 23 NHP studies (Table 1 and Supplementary Tables 4 and 5), 15 evaluate vaccine efficacy. Eleven evaluated viral load by RT-PCR and six measured nAb post-challenge; challenge doses varied greatly. All studies reported a reduction in peak viral load (Log10) compared to control groups. Peak viral load was significantly higher in control groups compared to vaccinated groups post-challenge. Each reported significant protection including, where indicated, from lung pathology/pneumonia, though none attain sterilizing immunity except for the Gao PiCoVacc inactivated whole virus vaccine given as weekly doses for 3-weeks and in the Guebre-Xabier NVX-CoV2373 vaccine. A caveat of NHP challenge studies is the short time interval between immunization and challenge, 1–6 weeks being most common.

28The benefits of comparative evaluation of datasets within the context of a systematic review and meta-analysis are self-evident at a time when enthusiasm for news of vaccine ‘breakthroughs’ can lead to incomplete reports in absence of opportunity for careful, granular, data-led comparison. We considered datasets with respect to binding and neutralizing Ab, prior anti-vector Ab, T cell responses and protection up to March 22, 2021. In doing so, it is a given that the correlates of protection (CoP) for COVID-19—which immune parameter/value gives the clearest actual indicator of protection—are not agreed, though evidence is accruing for nAb titer as a key CoP,,.

29Safety evaluation has encompassed a range of potential concerns, from initial reactogenicity profile to downstream post-challenge ADE or type 2 lung pathology. The meta-analysis shows a relatively low incidence of local and systemic AE at immunization. It is premature to comment on ADE in SARS-CoV-2 trials and rollout, but thus far it appears not to be an issue. The hypothetical concern originates in macaque studies evaluating either an MVA-vectored vaccine expressing SARS-CoV-1 spike or a SARS-CoV inactivated vaccine, both which resulted in ADE-associated lung immunopathology. An additional mechanism of vaccine-dependent, post-challenge immunopathology is Th2-dependent eosinophilic lung infiltration. Th2 cytokine-dependent lung immunopathology following virus challenge has been noted in vaccines for SARS-CoV-1 and for MERS-CoV in mice, ferrets and NHP. Again, such effects would only become apparent on subsequent exposure to virus, but it is encouraging that T cell responses across all the vaccine platforms compared here show strong skewing to Th1 cytokines, and little Th2 polarization. While anaphylaxis has been a very rare SAE in rollout of the Pfizer-BioNTech BNT162b2 vaccine, this was likely not picked up in the smaller sample size of the Phase 3 trial and with any vaccine allergic individuals excluded,. European rollout has highlighted rare development of thrombotic thrombocytopenia following ChAdOx1 nCoV-19 and Ad26.COV2.S vaccination–; this was not a feature identified in Phase III trials.

30This review has considered several inactivated virus vaccines, favored for the long track record of the approach (including polio vaccination) and diversity of viral antigens presented, though often considered to elicit less durable immunity. The Lin et al inactivated SAR-CoV1 vaccine elicited high nAb titers with a favourable safety profile, while the similar approach used for inactivation of SARS-CoV-2 also leads to a vaccine which is immunogenic and protective in NHP short-term challenge,.

31Among the many adenoviral-vector based approaches are the ChAdOx, AdH-5 and AdH26/5 vectored vaccines. ChAdOx vectored vaccines have been clinically tested over several years and across diverse infections–, so that there is a very expansive dataset on the ability to elicit robust Ab and, especially, strong T cell responses, including in the elderly and immunosuppressed. The choice of the chimpanzee viral vector to a considerable degree mitigates the confounder of prior Ab to limit the vaccine. Frequencies of virus-specific T cells induced by the ChAdOx vectors appear in most cases considerably higher than mean T cell responses induced by natural infection, though the extent to which this is a direct CoP remains to be seen. Of others considered here, the MVA-MERS-CoV vector vaccine produced low levels of nAb compared to other candidates. The mRNA-1273-SARS-CoV-2 vaccine elicited the highest Ab responses of any vaccine after a single dose, with a significant increase upon boosting,. Th1 cytokine responses were also strong, though requiring a boost. The Folegatti, ChAdOX-SARS-CoV-2 vaccine elicited strong Ab and T cell responses after a single injection, increasing further after boost. At present, there is no data to suggest that Th2 immunopathology will be a significant risk factor for SARS-CoV-2 vaccines. However, ongoing scrutiny will be needed to exclude the possible risk of Th2 immunopathology.

32The finding that some vaccines require a second boost for optimal responses while others may not is an important factor for decision-making. The global roll-out of effective vaccines demands unprecedented logistics to achieve coverage for a first round, and all the harder for a second round. At the time of writing, Phase 3 trial data are awaited from the Janssen Ad26.COV2.S Covid-19, which encompasses a 1-dose protocol.

33Prior immune memory has been a significant hurdle limiting the immunogenicity of adenoviral and other virus-vectored vaccines. Substantial pre-existing nAb responses were identified to the Zhu, AdH-5-SARS-CoV-2 vaccine,. Approximately 50% of those vaccinated were positive for pre-existing nAb >1:200, associated with a lower response to vaccination. Pre-immunity has implications for the geographic utility of vaccines, as certain regions have higher rates of specific disease which cause pre-immunity.

34Rhesus macaques develop pneumonia and other clinical aspects of SARS-CoV-2 like disease, as well as Ab and T cell responses similar to that seen in humans. Each of the tested candidates showed promising efficacy in NHP challenge, with caveats that these studies use greatly differing challenge doses making direct comparison difficult, and also that there tends to be a very short interval between priming and challenge. NHP studies, along with human challenge studies, will continue to play a key role for vital points that are hard to deduce from other approaches: ability of vaccines to achieve sterilizing immunity and delineation of CoP.

35It was difficult to attain across the board comparison of T cell immunity due to the different assays reported. Similarly, a lack of standardised antibody binding and neutralizing antibody testing means cross-comparison of binding antibody and nAb titers is limited by inter-assay variability. Efforts by the World Health Organisation to develop an international standard for SARS-CoV-2 neutralization assay calibration may go some way towards resolving this issue. Across vaccine platforms there are trade-offs between Ab titer, T cell frequency, reactogenicity and efficacy, with no single platform uniformly outstanding. RNA-based approaches have proved highly successful, though they can show a disadvantageous AE profile at higher doses and require a boost for maximal efficacy. The adenovirus-based approaches, especially ChAdOx-SARS-CoV-2, are of interest for their ability to induce exceptionally high T cell responses, probably higher than from natural infection, though this may prove less decisive if it transpires that nAb titer is of overriding significance as a CoP. NVX-CoV2373 strategy for spike protein with adjuvant bodes well for a simple approach to attaining exceptionally high, effective high nAb titers and strong protection in NHP challenge studies,.

36There were limitations inherent in this study: while new SARS-CoV-2 vaccine data is being produced rapidly, we have had to impose a finite search window, excluding some candidates that have received attention from press releases and preprints. Gaps in reporting of precise data in some studies prevented the completion of an effective meta-regression analysis of study metrics and determinants of vaccine safety, immunogenicity and efficacy. The lack of control groups in some human trials resulted in a limited meta-analysis of AE.

37At a time when we have already witnessed the successful development and roll-out of several COVID-19 vaccines, this review of 23 NHP and 32 human studies offers a template for high-granularity appraisal of the detailed metrics. To fully match the immune studies to fine-tuning of efficacy will require a better understanding of the CoP, particularly, relative importance of nAb and CD4 + responses. However, the front-running vaccines show excellent induction of nAb and T cell responses, and the associated ability to substantially limit severe disease, hospitalisation and death.

38Vaccine efficacy against emerging variant strains may, however, be diminished necessitating that vaccines be adapted to act specifically against variant strains and rapidly rolled out. There may also be opportunities to develop heterologous prime boost immunization schedules across vaccine platforms, potentially offering better immune responses and protection. The emergence of variant strains makes rapid mass rollout of high efficacy vaccines essential to reduce any selective advantage. Furthermore, as contingency plans develop, addressing the possible requirement for periodic boosters priming immunity targeted to variant sequences, it will be ever more important to have facilitated protocols that can intersperse use of different platforms, minimizing potential for limitation by anti-vector immunity.

39This systematic review and meta-analysis were conducted in line with PRISMA guidelines. An electronic search of PubMed and EMBASE databases was carried out using search terms and phrases related to HCoV vaccine candidates (Supplementary Table S1). Inclusion and exclusion criteria were defined to ensure all relevant studies were identified (Supplementary Table S2). Studies or trials evaluating the safety, immunogenicity or efficacy of a coronavirus vaccine candidate in humans or NHP were included. The need to impose a defined search period and also to exclude non-peer-reviewed preprints means that there are a large number of known vaccine candidates in Phase I/II or III trials that have been widely discussed in the press, but are not included in this review. The data included is updated to March 22, 2021 This study is registered with PROSPERO: CRD4202019030.

40Study selection was performed following pre-determined inclusion and exclusion criteria as described (Fig. 1 and Supplementary Table 2) extracting vaccine characteristics: platform, insert, route of administration, doses and schedule. Characteristics of challenge in NHP studies were also recorded: challenge virus, dose and swab location. For clarity in the Results and Discussion, we discuss specific, named studies by referring to the first author of the study and the shorthand name for the tested vaccine (Table 1 and Supplementary Table 5). Data on local and systemic AE for each of the studies were obtained.

41Geometric mean titer (GMT) by enzyme-linked immunosorbent assay (ELISA; IgG, IgG subtypes or IgA) and neutralization assays were extracted. Mean spot-forming cells per million peripheral blood mononuclear cells (SFC/million PBMC) were extracted for T cell ELISpots. If numerical Ab or T cell response datasets were not available, estimates were derived from respective graphs. Intracellular cytokine staining (ICS) data was obtained to analyze T cell responses, where described. Regarding vaccine efficacy analysis from NHP studies, challenge dose, viral strain, peak viral load, reverse transcription polymerase chain reaction (RT-PCR) swab location, RT-PCR positivity data and neutralizing Ab titers post-challenge were collated. To analyze human vaccine efficacy, the efficacy endpoints and overall vaccine efficacy (%) were extracted from relevant studies.

42RevMan 5.2 and GraphPad Prism were used for statistical analysis. With respect to AE analysis, Forest plot analysis was used to look at interstudy heterogeneity for AEs. Results were reported as risk ratios (RR) comparing the incidence of an AEs in experimental groups to control groups. A random effects, Mantel–Hanzeal model (95% CI) was used to determine effect sizes between studies. Statistical heterogeneity was assessed using I2 statistics: an I2 value of >50%; P < 0.05 was considered to represent severe heterogeneity, I2 value of 30–50%; P < 0.05 was considered to represent moderate heterogeneity and <30%; P < 0.05 was deemed to constitute insignificant heterogeneity. Since many first-in-human studies lacked control groups, descriptive analysis of AE was conducted. A grade three AE is here considered a severe AE. Serious AE (Grade 4) recorded in human studies were presented in tabular form.

43Due to the existence of multiple vaccine groups in each study, all vaccine groups were considered a statistical unit. Vaccine groups are considered to be experimental and control groups in the human and NHP trials. If a single dose of a vaccine candidate was used, it is defined as an unspecified dose.

44To evaluate interstudy Ab responses, we analyzed the GMT and nAb titer from ELISA and neutralizing assays respectively. There was a lack of comprehensive numerical datasets of Ab titers across human studies; in some cases Ab titers were estimated from graphical representations. Ab titers analyzed were collected circa 28 d after a given injection, allowing for inclusion of Ab titer data from vaccine regimens involving prime/boosts. Transformed mean log Ab titers (Log10) were estimated from human and NHP studies to determine statistical difference between groups using an unpaired t test (95% CI).

45T cell responses were evaluated by extracting IFNγ ELISpot data. In some cases, estimated frequencies were extracted from graphs. T cell responses were extracted circa 28 d after individual immunizations and represented by Forest plot to show interstudy differences in ELISpot SFC/million PBMC. Transformed log mean T cell responses were obtained from human and NHP studies to test statistical differences between NHP and human groups.

46For NHP efficacy analysis, we analyzed differences between control and experimental peak viral loads and nAb responses (Log10) post-challenge to examine vaccine-induced protection.

47Further information on research design is available in the Nature Research Reporting Summary linked to this article.