The rapid growth of the consumer credit card market has introduced substantial regulatory and risk management challenges. To address these challenges, financial institutions increasingly adopt advanced machine learning models to improve default prediction and portfolio monitoring. However, the use of such models raises additional concerns regarding transparency and fairness for both institutions and regulators. In this study, we investigate the consistency of Shapley Additive Explanations (SHAPs), a widely used Explainable Artificial Intelligence (XAI) technique, through a case study on credit card probability-of-default modeling. Using the Default of Credit Card dataset containing 30,000 consumer credit accounts information, we train 100 Extreme Gradient Boosting (XGBoost) models with different random seeds to quantify the consistency of SHAP-based feature attributions. The results show that the feature SHAP stability is strongly associated with feature importance level. Features with high predictive power tend to yield consistent SHAP rankings (Kendall’s W = 0.93 for the top five features), while features with moderate contributions exhibit greater variability (Kendall’s W = 0.34 for six mid-importance features). Based on these findings, we recommend incorporating SHAP stability analysis into model validation procedures and avoiding the use of unstable features in regulatory or customer-facing explanations. We believe these recommendations can help enhance the reliability and accountability of explainable machine learning framework in credit risk management.
Lin et al. (2025) studied this question.