Implications for future pandemics from a living systematic review and critical evaluation of observational studies of the effectiveness of COVID-19 vaccination against the Omicron variant

Reliable evidence on COVID-19 vaccine effectiveness during the period of Omicron variant predominance was essential to inform global vaccination strategies. Our living systematic review, commissioned by the World Health Organization (WHO), was established to provide an ongoing synthesis and critical appraisal of comparative observational studies evaluating the effectiveness of vaccines against COVID-19/SARS-CoV-2 infection caused by the Omicron variant. Weekly searches covering seven databases up to 31 December 2022 sought evidence for WHO-approved vaccine platforms, including mRNA vaccines (BNT162b2, mRNA-1273), adenoviral vector vaccines (ChAdOx1-S, SII-ChAdOx1 nCoV-19, Ad26.COV2.S, Gam-COVID-Vac) and inactivated virus vaccines (CoronaVac, BBV-152). We used the Risk Of Bias In Non-randomized Studies - of Interventions (ROBINS-I) tool to assess the validity of estimates of vaccine effectiveness. The review included 79 studies of over 53 million individuals, contributing 2032 estimates of vaccine effectiveness against asymptomatic SARSCoV- 2 infection (36; 45.6%), symptomatic COVID-19 (31; 39.2%), severe or critical COVID-19 (53; 67.1%) and all-cause mortality (2; 2.5%). Structured comparisons of vaccine performance across platforms, dosing schedules and populations, focussed on severe or critical COVID-19 disease. Vaccination was observed to be strongly protective against severe or critical disease caused by the Omicron variant, particularly for mRNA vaccines. Most results were judged to be at moderate risk of bias (361, 18%) or at serious risk of bias (1591, 78%) with the principal concern being confounding. We discuss implications for future pandemic preparedness, including the importance of standardized study protocols; robust data linkage between vaccination, testing and outcomes; and validated risk-of-bias frameworks for diverse observational designs.