The integration of artificial intelligence (AI) in medicine has applications across several clinical domains, spanning from disease prevention and diagnosis through treatment and long-term care, as well as remote care. However, many AI systems are inherently characterized by limited explainability, meaning the processes behind their outcomes cannot be clearly understood or communicated to humans, whether developers or end users. This viewpoint explores the importance of AI explainability in medicine by first tracing its evolution from a primarily ethical concern to a legal requirement. It then examines the connection between explainability and the trustworthiness of AI systems. Finally, it considers how explainability is approached from a technical standpoint and its inherent tension with achieving high accuracy.
Sacchi et al. (Wed,) studied this question.