A qualitative study on XAI techniques for Software Defect Prediction

Context: Machine learning (ML) models are increasingly used in Software Defect Prediction (SDP) to identify defect-prone software modules. However, many ML models operate as black boxes, making their predictions difficult for developers to interpret and trust. Although Explainable Artificial Intelligence (XAI) techniques such as LIME, SHAP, and BreakDown are widely adopted to improve transparency, different explainers often produce inconsistent and conflicting explanations for the same prediction outcomes. Objective: This study investigates the use of XAI techniques in SDP, evaluates the consistency and reliability of commonly used explainers, examines practitioners’ challenges in interpreting explanations, and explores strategies for improving explanation trustworthiness and usability. Method: We conducted a mixed-methods study consisting of four phases: (1) a systematic literature review of 93 studies on XAI in SDP; (2) a controlled comparative evaluation of multiple explainability techniques, including LIME, SHAP, PyExplainer, PDP, ICE, and BreakDown; (3) a scenario-based analysis of explanation behavior using JIRA-based defect datasets and Random Forest prediction models; and (4) a practitioner survey involving 71 participants to investigate interpretability, usability, and trust-related concerns. Explanation consistency was evaluated using Feature Agreement (FA), Rank Agreement (RA), and Sign Agreement (SA), supported by statistical validation using Friedman ranking and Kendall Tau correlation analysis. Results: The results reveal substantial disagreement among explainers in terms of feature importance, feature ranking, and feature influence direction. Statistical analysis further confirms significant variation in explanation consistency across different techniques, with SHAP demonstrating comparatively stronger agreement behavior. The practitioner study also identified several practical challenges, including low trust, contradictory explanations, and limited integration support in software engineering workflows. Conclusion: Current XAI techniques for SDP still face important reliability and interpretability challenges that limit their practical adoption. Our findings highlight the need for agreement-aware evaluation, explanation stability analysis, and developer-centered explainability support to improve the trustworthiness and usability of XAI systems in software engineering.