XAI: On Explainability and the Obligation to Explain

Abstract The increasing relevance of AI systems paired with their repeatedly observed opacity gave rise to the field of explainable artificial intelligence (XAI). Methods of XAI are being developed and evaluated based on whether they overcome said opacity by providing explanations, thereby apparently pursuing an epistemic end. What is commonly sidestepped, however, is the distinction between the ability and the obligation to explain: In which specific cases and under what specific circumstances is there a moral obligation to give an explanation? We address this issue by first arguing that apart from their epistemic virtues, explanations can be ethically relevant and hence subject to moral considerations. Further, we build on Kantian theories to provide a framework that systematizes in which instances there is a moral obligation to explain and in which instances there is none. We relate our framework both to Forst’s “right to justification” and to the “right to explanation” as codified in recent legislation. Finally, we show that our framework does not only close a gap in clarifying when explanations of AI systems are required, but that it also paves the way for concrete suggestions regarding the design of XAI methods.