Comparative analysis of classical and Bayesian optimisation techniques: Impact on model performance and interpretability in credit risk modelling using SHAP and PDPs

Financial institutions often hesitate to use complex models such as random forests and extreme gradient boosting (XGBoost) for credit risk assessments due to challenges in selecting the optimal hyperparameters and the interpretability of these 'black box' models. This study addresses these issues by comparing the effects of classical grid search and Bayesian hyperparameter optimisation techniques on models' performance and interpretability in credit risk modelling. The results indicate that Bayesian optimisation improves recall for XGBoost, making it more effective at identifying defaulters while offering no notable advantage for random forests and even reducing performance for logistic regression. Although Bayesian optimisation reduces the computational time required for finding optimal hyperparameters, it does not improve the models' discriminatory power (area under the curve). The findings also suggest that different optimisation techniques influence feature importance and ranking, with SHapley Additive exPlanation (SHAP) values revealing that slight hyperparameter adjustments can lead to substantial changes in feature importance, particularly for logistic regression. However, partial dependence plots (PDP) for the variable Rate under Bayesian and classically optimised models are similar, indicating that using classical or Bayesian optimisation techniques does not alter the relationship between features and default probability. These insights emphasise the importance of selecting an appropriate optimisation approach to balance models' performance and explainability, with significant implications for credit risk modelling decisions.