Integrating human and artificial intelligence for robust postmarketing safety surveillance systems: reflections from the FDA Sentinel Innovation Center

Objectives To describe considerations for integration of human and artificial intelligence for creating a postmarketing surveillance system capable of timely and reliably identifying causal effects of medications on safety endpoints. Materials and Methods The FDA has prioritized more extensive Electronic Health Records (EHR) integration along with generative artificial intelligence and machine learning (Gen AI/ML) into the national active surveillance program for medical products—the Sentinel Initiative. Based on our experience of leading these efforts, we provide perspectives on the opportunities and challenges of Gen AI/ML integration into Sentinel. Results Using specific examples, we outline the role of Gen AI and ML in a causal inference framework for scalable information extraction, assessment of fitness-for-purpose of data sources, diagnosing residual confounding, and enhancing confounding adjustment. Critically, we outline steps and checkpoints along the way where human involvement remains indispensable. Discussion and conclusion In public health applications where stakes are high, use of Gen AI/ML needs to be carefully considered with appropriate guardrails ensuring human expert involvement.