COVID-19 vaccine efficacy and effectiveness—the elephant (not) in the room

Elephant in the room
Photo by Joaquín Rivero on Unsplash

Piero Olliaro, Els Torreele, Michel Vaillant.

The Lancet, Open Access, April 20, 2021.

Approximately 96 COVID-19 vaccines are at various stages of clinical development.1 At present, we have the interim results of four studies published in scientific journals (on the Pfizer–BioNTech BNT162b2 mRNA vaccine,2 the Moderna–US National Institutes of Health [NIH] mRNA-1273 vaccine,3 the AstraZeneca–Oxford ChAdOx1 nCov-19 vaccine,4 and the Gamaleya GamCovidVac [Sputnik V] vaccine)5 and three studies through the US Food and Drug Administration (FDA) briefing documents (on the Pfizer–BioNTech,6 Moderna–NIH,7 and Johnson & Johnson [J&J] Ad26.COV2.S vaccines).8 Furthermore, excerpts of these results have been widely communicated and debated through press releases and media, sometimes in misleading ways.9 Although attention has focused on vaccine efficacy and comparing the reduction of the number of symptomatic cases, fully understanding the efficacy and effectiveness of vaccines is less straightforward than it might seem. Depending on how the effect size is expressed, a quite different picture might emerge (figureappendix).

Figure RRR and NNV with 95% CI ranked by attack rate in the unvaccinated (placebo) group for five COVID-19 vaccines. Show full caption

View Large Image | Figure Viewer | Download Hi-res image| Download (PPT)

Vaccine efficacy is generally reported as a relative risk reduction (RRR). It uses the relative risk (RR)—ie, the ratio of attack rates with and without a vaccine—which is expressed as 1–RR. Ranking by reported efficacy gives relative risk reductions of 95% for the Pfizer–BioNTech, 94% for the Moderna–NIH, 91% for the Gamaleya, 67% for the J&J, and 67% for the AstraZeneca–Oxford vaccines. However, RRR should be seen against the background risk of being infected and becoming ill with COVID-19, which varies between populations and over time. Although the RRR considers only participants who could benefit from the vaccine, the absolute risk reduction (ARR), which is the difference between attack rates with and without a vaccine, considers the whole population. ARRs tend to be ignored because they give a much less impressive effect size than RRRs: 1·3% for the AstraZeneca–Oxford, 1·2% for the Moderna–NIH, 1·2% for the J&J, 0·93% for the Gamaleya, and 0·84% for the Pfizer–BioNTech vaccines.

Supplementary Material

Supplementary appendix Download .pdf (.22 MB)

References

  1. 1.
    • Zimmer C 
    • Corum J 
    • Wee S-L
    Covid-19 Vaccine Tracker.https://www.nytimes.com/interactive/2020/science/coronavirus-vaccine-tracker.htmlDate accessed: March 10, 2021View in Article 
  2. 2.
    • Polack FP 
    • Thomas SJ 
    • Kitchin N 
    • et al.
    Safety and efficacy of the BNT162b2 mRNA COVID-19 Vaccine.N Engl J Med. 2020; 383: 2603-2615View in Article 
  3. 3.
    • Baden LR 
    • El Sahly HM 
    • Essink B 
    • et al.
    Efficacy and safety of the mRNA-1273 SARS-CoV-2 Vaccine.N Engl J Med. 2021; 384: 403-416View in Article 
  4. 4.
    • Voysey M 
    • Clemens SAC 
    • Madhi SA 
    • et al.
    Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK.Lancet. 2021; 397: 99-111View in Article 
  5. 5.
    • Logunov DY 
    • Dolzhikova IV 
    • Shcheblyakov DV 
    • et al.
    Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia.Lancet. 2021; 397: 671-681View in Article 
  6. 6.
    • US Food and Drug Administration
    Vaccines and Related Biological Products Advisory Committee meeting: FDA briefing document.https://www.fda.gov/advisory-committees/advisory-committee-calendar/vaccines-and-related-biological-products-advisory-committee-december-10-2020-meeting-announcementDate: Dec 10, 2020Date accessed: March 10, 2021View in Article 
  7. 7.
    • US Food and Drug Administration
    Vaccines and Related Biological Products Advisory Committee meeting: FDA briefing document.https://www.fda.gov/advisory-committees/advisory-committee-calendar/vaccines-and-related-biological-products-advisory-committee-december-17-2020-meeting-announcementDate: Dec 17, 2020Date accessed: March 10, 2021View in Article 
  8. 8.
    • US Food and Drug Administration
    Vaccines and Related Biological Products Advisory Committee meeting: FDA briefing document.https://www.fda.gov/advisory-committees/advisory-committee-calendar/vaccines-and-related-biological-products-advisory-committee-february-26-2021-meeting-announcementDate: Feb 26, 2021Date accessed: March 10, 2021View in Article 
  9. 9.
    • Olliaro P
    What does 95% COVID-19 vaccine efficacy really mean?.Lancet Infect Dis. 2021; (published online Feb 17.)https://doi.org/10.1016/S1473-3099(21)00075-XView in Article 
  10. 10.
    • Brown RB
    Outcome reporting bias in COVID-19 mRNA vaccine clinical trials.Medicina (Kaunas). 2021; 57: 199View in Article 
  11. 11.
    • Dagan N 
    • Barda N 
    • Kepten E 
    • et al.
    BNT162b2 mRNA COVID-19 vaccine in a nationwide mass vaccination setting.N Engl J Med. 2021; (published online Feb 24.)https://doi.org/10.1056/NEJMoa2101765View in Article 

Be the first to comment on "COVID-19 vaccine efficacy and effectiveness—the elephant (not) in the room"

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.