Near misses analysis (NMA): A new explainable AI approach for model understanding, comparison, debugging and adversarial attack detection

This paper introduces a novel explainable artificial intelligence (XAI) approach based on near-misses analysis (NMA). This approach uses the network close related predictions to reveal a hierarchy of logical concepts inferred from the latent decision-making process of a neural network (NN) without delving into its explicit structure. Several NMA usage possibilities are reported in this paper. First, it serves to create an explanation in the form of a gradually expanding explicit linked concepts which coupled with a proper dictionary can provide a scoring method to differentiate which of any given models is better at providing human-like conceptual explanations. In addition, NMA can be used to pinpoint how to improve models according to their explanatory outcome. Finally, it enables to detect adversarial attacks. The proposed XAI approach is examined on different network architectures that vary in size and shape (ResNet, VGG, EfficientNet, MobileNet) and on several datasets which were already organized in a hierarchical concept structure (ImageNet, CIFAR100). Results demonstrate that NMA can reflect NNs latent concepts generation process. Moreover, using the devised scoring method, it is reported that efficient architectures, which achieve a similar accuracy level with less neurons, may still pay the price of explainability and robustness. • Introduce a new XAI method based on neural networks concepts generation. • Demonstrate the explanatory capabilities of this method on various models and datasets. • Propose a new explanatory scoring method. • Demonstrate how to use the new method for model debugging. • Demonstrate how to use the new method for detecting adversarial attacks.