Fighting the disagreement in Explainable Machine Learning with consensus

Abstract While the predictive accuracy of machine learning (ML) models is often their most lauded attribute, in many scientific domains, the transparency of the decision-making process is equally crucial. To demystify the decision logic of ML models, interpretability algorithms are employed. Yet, the diversity of available algorithms frequently leads to a lack of consensus, resulting in disparate and sometimes contradictory model explanations. Addressing this challenge, consensus functions are utilized post hoc to harmonize these interpretations. However, the reliability of the synthesized explanation remains contingent on the choice of consensus function, among other variables. In this study, five ML models were systematically trained on four synthetic datasets with pre-established ground truths. Subsequent explanations were generated using a suite of both local and global model-agnostic interpretability algorithms. Consensus was then derived using five established functions and a novel function created by the researchers that integrates model accuracy into the consensus-building process. The innovative function demonstrated superior accuracy in pinpointing pertinent features, reinforcing the notion that an aptly crafted consensus function is instrumental in resolving discrepancies within model explainability.