Evaluating the necessity of the multiple metrics for assessing explainable AI: A critical examination

This paper investigates the specific properties of Explainable Artificial Intelligence (xAI), particularly when implemented in AI/ML models across high-stakes sectors, in this case cybersecurity. The authors execute a comprehensive systematic review of xAI properties, various evaluation metrics, and existing frameworks to assess their utility and relevance. Subsequently, the experimental sections evaluate selected xAI techniques against these metrics, delivering key insights into their practical utility and effectiveness. The findings highlight that the proliferation of metrics enhances the understanding of xAI systems but simultaneously exposes challenges such as metric duplication, inefficacy, and confusion. These issues underscore the pressing need for standardized evaluation frameworks to streamline their application and strengthen their effectiveness, thereby improving the overall utility of xAI in critical domains.