Joint Optimization of the Decision Threshold and Hyperparameters of Gradient Boosting for Cost Reduction in Customer Churn Prediction

Retention campaigns depend on identifying the customers most likely to discontinue service, yet churn classifiers are usually deployed with the conventional decision threshold of one half, a rule that ignores both the rarity of churn and the unequal costs of the two classification errors. This study proposes a framework that jointly optimizes the hyperparameters of a gradient boosting learner and its decision threshold under an explicit misclassification cost, in which the rime-ice optimization algorithm (RIME) searches the hyperparameters while the cost-minimizing threshold is computed exactly for every candidate configuration. The framework is embedded in a nested protocol of five repetitions of two-fold cross-validation and evaluated on 50,000 customers of the French telecommunications operator Orange, released for the KDD Cup 2009, of whom 7.34% churned. With a missed churner costing ten times an unnecessary retention offer, the framework reduced the expected cost per customer of CatBoost from 0.7231 to 0.5509, of XGBoost from 0.7139 to 0.5537, and of LightGBM from 0.7137 to 0.5551, relative to each learner in its default configuration at the conventional threshold; all three reductions are significant under the combined five-by-two cross-validation F-test with Holm correction. The tuned CatBoost model attained the lowest cost, an area under the receiver operating characteristic curve of 0.7272, and the highest F1 score for the churn class, 0.2488. Ablation analyses locate the source of this improvement. For CatBoost, optimizing the threshold of the default configuration alone accounts for 0.1682 of the total reduction of 0.1722, so cost-sensitive threshold selection is the dominant component and the additional hyperparameter search contributes a small further gain; for XGBoost and LightGBM, whose default configurations rank customers less well, the search contributes a larger share. A transfer analysis shows that applying the threshold selected on the inner selection set to the refitted model costs 0.0080 per customer relative to the threshold that would be optimal on the assessment data itself. No statistically significant differences were detected among six search strategies under the common evaluation budget.