Explainable AI for Early Sprint Delivery Risk Prediction Across Agile Software Projects

Artificial intelligence (AI) can support early identification of Sprint delivery problems in Agile software development, but predictive performance alone is insufficient when model outputs inform human judgement. This study investigates explainable artificial intelligence (XAI) for predicting Sprint delivery underachievement across five open-source project families—Apache, JBoss, JIRA, MongoDB, and Spring—using software-development traces. Using 3834 board–Sprint observations from 1536 logical Sprint groups, we compare logistic regression, random forest, XGBoost, and muti-layer peceptron (MLP) against a naïve prior-probability baseline under group-aware within-project and leave-one-project-out cross-project validation. We evaluate Strict 0% and Extended 0% predictors from the published 0% snapshot. Within-project discrimination, measured by area under the receiver operating characteristic curve (ROC-AUC), exceeded cross-project performance for all trained models; the highest macro-averaged values were 0.722 for the Strict 0% MLP and 0.784 for the Extended 0% random forest. However, the operational availability of the additional issue-derived predictors in Extended 0% at Sprint start could not be independently verified. Sensitivity analyses showed that model rankings changed across alternative specifications. Shapley additive explanations (SHAP) identified vel_starttime as the most influential feature overall in both selected models, although feature rankings varied across held-out projects. Predictions and explanations are intended as inspectable decision-support information, while human decision-makers retain final judgement and action. Effects on decision-making performance and quality require future user-centred empirical validation.