This comprehensive literature review explores the latest advancements in explainable artificial intelligence (XAI) techniques within the field of medical imaging (MI). Over the past decade, machine learning (ML) and deep learning (DL) technologies have made significant strides in healthcare, enabling advancements in tasks such as disease diagnosis, medical image segmentation, and the detection of various medical conditions. However, despite these successes, the widespread adoption of AI-driven tools in clinical practice remains slow, primarily due to the “black-box” nature of many AI models. These models make decisions without transparent reasoning, which poses significant barriers in critical medical and legal environments, where accountability and trust are paramount. This review investigates various XAI methods, focusing on both intrinsic and post-hoc techniques, to evaluate their potential in addressing these challenges. The paper examines how XAI can enhance the transparency of healthcare algorithms, thereby fostering greater trust and confidence among clinicians, patients, and regulators. Key challenges faced by XAI in healthcare, such as limited interpretability, computational complexity, and the absence of standardized evaluation frameworks, are discussed in detail. Furthermore, this work highlights existing gaps in the literature, including the lack of detailed comparative analyses of specific XAI techniques, especially in terms of their mathematical foundations and applicability across diverse medical imaging contexts. In response to these gaps, the paper introduces a new set of standardized evaluation metrics aimed at assessing XAI performance across various medical imaging tasks, such as image segmentation, classification, and diagnosis. The review proposes actionable recommendations for enhancing the effectiveness of XAI in healthcare, with a focus on real-world clinical applications. Unlike previous studies that focus on broader overviews or limited subsets of methods, this work provides a comprehensive comparative analysis of over 18 XAI techniques, emphasizing their strengths, weaknesses, and practical implications. By offering a detailed understanding of how XAI methods can be integrated into clinical workflows, this paper aims to bridge the gap between cutting-edge AI technologies and their practical use in medical settings. Ultimately, the insights provided are valuable for researchers, clinicians, and industry professionals, encouraging the adoption and standardization of XAI practices in clinical environments, thus ensuring the successful integration of transparent, interpretable, and reliable AI systems into healthcare.
Fayaz et al. (Mon,) studied this question.