Metrics and Quality Attributes of AI-Based Software as a Medical Device

Software as a Medical Device (SaMD) refers to software designed for medical purposes that operates independently of any physical hardware. Artificial Intelligence (AI)-based SaMD refers to standalone software utilizing Machine Learning (ML) and other AI techniques to perform medical functions without being part of a physical device. With the growth in research and practical applications of AI-SaMD, a variety of quality attributes have been proposed. However, the existing body of work lacks a comprehensive and cohesive review from a quality assurance perspective. This research paper aims to fill this gap by conducting a systematic analysis and synthesis of literature published between 2015 and 2024 to identify commonly addressed quality attributes, the metrics used to evaluate them, and emerging directions for future research. The identified key quality attributes included accuracy, AI model transparency, safety, performance, and Specificity and Sensitivity (S&S) as the most frequently used criteria for assessing the quality of AI-SaMD. The findings highlight significant gaps in the current research landscape, particularly the absence of a universal metric to comprehensively evaluate the quality of AI-SaMD. Furthermore, there is a lack of alignment between identified quality attributes, their corresponding metrics, and clinical validation standards, such as Food and Drug Administration (FDA) guidelines. The study concludes by offering valuable insights for stakeholders, including clinicians, technologists, and policymakers, and by outlining promising avenues for future research.