Explainable artificial intelligence (XAI) is employed to clarify the rationale behind AI outputs and resolve black-box nature of artificial intelligence (AI). This is intended to enhance trustworthiness and usability of AI-based technology for operation or decision-making support. However, XAI explanations now are often more accessible to developers, who construct, verify, and optimize AI models, than to operators, who need to understand and employ these models for decision-making. Therefore, this study aims to develop a Grad-CAM-based deep learning methodology that provides operator-centered explanations for enhancing explainability of AI outputs and the trustworthiness of the AI technologies for operation support in the context of performing procedure-based operating tasks. In this study, an XAI model based on gradient-weighted class activation mapping and a dilated causal convolutional neural network was developed to identify abnormal states and provide operator-centered explanations within the scope of abnormal operating procedures corresponding to identified abnormal states. Furthermore, representation of operator-centered explanations was addressed to effectively display the AI-supported information from the proposed model on human-system interface for operating tasks.
Koo et al. (Sun,) studied this question.