During the pandemic, the accelerated review and authorization of coronavirus disease 19 (COVID-19) vaccines by regulatory authorities elicited the need for rapid and thorough worldwide signal detection and evaluation. To meet this need, the European Medicines Agency and other health authorities expected that, in addition to routine signal detection, COVID-19 vaccine manufacturers should leverage observed-to-expected (O/E) analyses unconventionally as a quantitative method for signal detection of adverse events of special interest (AESIs). The objective of O/E analyses in vaccine signal detection was to determine if AESIs were occurring at a higher-than-expected rate in the vaccinated population in comparison with an unexposed population. The use of O/E was intended to mitigate the challenge of analyzing large volumes of individual case safety reports (ICSRs) received over a very short period following mass vaccination campaigns. The “Beyond COVID-19 Monitoring Excellence” (BeCOME) initiative, a non-competitive voluntary initiative launched in 2022 by COVID-19 vaccine Marketing Authorization Holders (MAHs) and key stakeholders, was established to align systems, enhance processes, and foster innovation in post-marketing vaccine monitoring, building on lessons from the pandemic. A dedicated working group was created to review and share MAHs’ experience on O/E analyses used as an additional tool for signal detection during the COVID-19 pandemic. This review presents the industry perspective on using O/E analyses for COVID-19 vaccine signal detection, including challenges and limitations encountered, and proposes best practices for future improvement. Despite the priority and resources devoted to O/E analyses, no de novo signals resulting in the identification of safety concerns were detected using this methodology during the COVID-19 pandemic. O/E analyses are most useful when source data are accurate and there is a high level of confidence in the assumptions and parameters used. In the context of the COVID-19 pandemic, confidence in certain assumptions and parameters was low, limiting the value of O/E analyses in signal detection. Nevertheless, O/E analyses applied for signal refinement, as traditionally used, proved to be useful. Industry experiences support maintaining O/E analyses as a tool for signal refinement and standardizing methodological approaches as much as possible to enhance its future application and comparability across stakeholders.
Fretta et al. (Mon,) studied this question.