Systematic review reveals omission of explainable AI in 58.2% of AI medical device studies, indicating that opacity poses under-reported safety risks.
While AI-enabled medical devices (AIeMD) are redefining healthcare, safety relies on human-AI interaction as much as algorithmic performance. This systematic review of 53 studies reveals that 58.2% of research omits Explainable AI (XAI) methods, despite transparency being a regulatory and safety necessity. Results identify a "symmetry of modality": qualitative interviews correlate with written text explanations, while Think-Aloud protocols better assess cognitively demanding tools like SHAP values. Currently, AI-specific risks like automation bias, driven by algorithmic opacity, are critically under-reported. To ensure safe adoption, practitioners must move beyond "user satisfaction" to treat transparency and XAI as safety-critical requirements for trust calibration.
No takes yet. Share an insight, caveat, or question.
Oliveira et al. (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: