Benefit-Risk Reporting for FDA-Cleared Artificial Intelligence−Enabled Medical Devices

Importance Devices enabled by artificial intelligence (AI) and machine learning (ML) are increasingly used in clinical settings, but there are concerns regarding benefit-risk assessment and surveillance by the US Food and Drug Administration (FDA). Objective To characterize pre- and postmarket efficacy, safety, and risk assessment reporting for FDA-cleared AI/ML devices. Design and Setting This was a cross-sectional study using linked data from FDA decision summaries and approvals databases, the FDA Manufacturer and User Facility Device Experience Database, and the FDA Medical Device Recalls Database for all AI/ML devices cleared by the FDA from September 1995 to July 2023. Data were analyzed from October to November 2024. Main Outcomes and Measures AI/ML reporting of study design, data availability, efficacy, safety, bias assessments, adverse events, device recalls, and risk classification. Results The analysis included data for all 691 AI/ML devices that received FDA clearance through 2023, with 254 (36.8%) cleared in or after 2021. Device summaries often failed to report study designs (323 46.7%), training sample size (368 53.3%), and/or demographic information (660 95.5%). Only 6 devices (1.6%) reported data from randomized clinical trials and 53 (7.7%) from prospective studies. Few premarket summaries contained data published in peer-reviewed journals (272 39.4%) or provided statistical or clinical performance, including sensitivity (166 24.0%), specificity (152 22.0%), and/or patient outcomes (3 lt;1%). Some devices reported safety assessments (195 28.2%), adherence to international safety standards (344 49.8%), and/or risks to health (42 6.1%). In all, 489 adverse events were reported involving 36 (5.2%) devices, including 458 malfunctions, 30 injuries, and 1 death. A total of 40 devices (5.8%) were recalled 113 times, primarily due to software issues. Conclusions and Relevance This cross-sectional study suggests that despite increasing clearance of AI/ML devices, standardized efficacy, safety, and risk assessment by the FDA are lacking. Dedicated regulatory pathways and postmarket surveillance of AI/ML safety events may address these challenges.