Evaluating the trustworthiness of black-box machine learning models remains a significant methodological challenge. Their lack of transparency and interpretability limits applicability, because stakeholders often seek transparency before trusting the results of black-box machine learning models. Explainable AI (XAI) methods provide for human-understandable justifications and informed decision-making of these black-box architectures. Therefore, it is imperative to select the proper XAI model tailored to specific tasks. In this research, we focus on examining four XAI techniques: PEEK, LRP, GRAD-CAM, and LIME to understand how they perform against each other for image classification tasks. We evaluate the performance, robustness, generalizability, noise stability, and computational efficiency of these methods using a globally recognized dataset. With 7390 images, the Oxford IIT pet dataset provides a comprehensive resource for training a custom Convolutional Neural Network (CNN) and VGG16, enabling a consistent evaluation of each XAI method. First, we analyze the saliency maps of the input images and observe the regions predicted by these XAI methods, and then leverage a noise analysis approach to evaluate their performance in terms of accuracy. We further explore the robustness, run-time, and “faithfulness” metrics of each XAI method. In general, we find that these methods can identify a set of input-data features that are critical for accurate classification but also intuitive, such as the outline, face, and eyes of subjects. However, our analysis reveals only marginal consensus among XAI methods in identifying those critical features. Grad-CAM demonstrates strong robustness and stability in VGG16, but the performance on the shallow CNN model remained inconsistent.
Chakraborty et al. (Mon,) studied this question.