Explainable throughout the AI lifecycle: the evolution of the explainability provisions in the EU AI Act

Explainability of AI systems is branded as a solution to the black box problem. The EU AI Act introduces AI explainability as a requirement for high-risk AI systems, including a right to explanation. However, understanding the wording of the requirements and the related provider and deployer obligations is complicated. This paper takes a legal-evolution research approach by analysing legal literature and the wording of the AI HLEG Guidelines, the OECD AI Principles, the AI Act, and the Council of Europe’s AI Convention, contributing to understanding the AI Act’s explainability requirements. First, this paper finds that explanations can take different forms while fulfilling various functions. It finds that, in the AI Act, the central functions of AI explanations are to understand the AI system’s inner workings and outputs, to enable contestation of decisions by affected persons, to facilitate usability by the deployer and to achieve legal compliance. Second, this research maps the explanations required under the AI Act onto the AI lifecycle phases. Resultantly, this research finds that the AI Act expects the provider and deployer to engage in continuous and iterative explainability by design, development and deployment throughout the AI lifecycle, following the evolution of their AI system.