What do we explain when explaining AI systems? A multi-layered conceptual framework for compliance analysis

The use of artificial intelligence (AI) systems is becoming increasingly prevalent in critical domains such as law and medicine. This intensifies the need to ensure effective compliance with ethical and legal values, particularly those related to transparency and explainability. However, the notion of explainability is often used as if AI systems can be discussed and explained on only one level, concerning the functioning of the system as a whole. It should be recognised that such systems are not unidimensional and, consequently, meaningful discussion about explainability is only possible if one first specifies what can and should be explained and how. The present theoretical and conceptual study is devoted to the decomposition of levels relating to what can be explained in a system (horizontal levels of explainability) and the levels at which explanation can occur (vertical levels). The horizontal levels include: (1) the problem, (2) the development of the system, and (3) the system itself. In addition, we distinguish between (2a) a static perspective and (2b) a dynamic perspective on the development of the system and between explaining (3a) the input of the system, (3b) its operations, and (3c) its output. Within the vertical levels, we propose a division according to two criteria: the level of abstraction and the degree of formalisation of knowledge. The presented approach considers the multi-layered nature of what and how can be explained in AI systems. The validity of the model is supported by three use cases: (1) mapping Article 13 of the AI Act, (2) analysing a debate surrounding COMPAS system, (3) analysing levels of opacity in LLM-based AI chatbots. The proposed approach offers enhanced conceptual clarity in discussions on AI explainability, while also providing a tool for more effective compliance analysis.