What question did this study set out to answer?

The objective is to systematically analyze various Gradient Boosting models to evaluate their predictive performance and stability across multiple datasets.

June 15, 2026Open Access

Gradient Boosting Revisited: Comparative Analysis of Selected Advances on Real-World Tabular Data

MAMoses Apambila AgebureNavrongo Health Research Centre JWJapheth WireduRegent University College of Science and Technology SAStephen AkobreNavrongo Health Research Centre

Key Points

The objective is to systematically analyze various Gradient Boosting models to evaluate their predictive performance and stability across multiple datasets.
Conducted a systematic analysis of seven models: XGBoost, LightGBM, CatBoost, HistGradientBoosting, GradientBoosting, AdaBoost, and MorphBoost.
Trained models on ten benchmark datasets with a fixed 80:20 train-test split and 3-fold cross-validation.
Measured performance using accuracy, F1-score, and ROC-AUC metrics.
CatBoost achieved the highest mean accuracy of 0.9400 and a near-perfect ROC-AUC of 0.9915.
HistGradientBoosting was identified as the most stable model, followed by LightGBM and XGBoost.
MorphBoost shows promise in binary and high-dimensional datasets but lacks support for multiclass handling.

Abstract

Gradient Boosting has become one of the approaches design to improve general predictive performance as well as overcome some speciﬁc learning challenges. Though mature, there are still new adaptive variants being created to enhance ﬂexibility, efﬁciency, as well as overall predictive power. However, there are limited benchmarking studies that sought to establish the generalisation abilities of these techniques especially the newer variants under varying conditions. This study, therefore, conducts a systematic analysis of seven Gradient Boosting models: XGBoost, LightGBM, CatBoost, HistGradientBoosting, GradientBoosting, AdaBoost, and the adaptive MorphBoost on ten benchmark datasets different challenges. All models were trained using a ﬁxed 80:20 train–test split, with 3-fold cross-validation performed solely on the training portion to estimate stability. Performance was measured using accuracy, F1-score, and ROC-AUC to guarantee fairness and reproducibility. The ﬁndings indicate that CatBoost produced the highest mean accuracy of 0.9400 and a near-perfect ROC-AUC of 0.9915, which means that it can effectively generalize across diverse data types. HistGradientBoosting is identiﬁed as the most stable model across datasets with a good level of performance and computational efﬁciency, and it is currently followed by LightGBM and XGBoost. MorphBoost shows promise on binary and high-dimensional datasets where its implementation is fully supported, though its current lack of native multiclass handling limits general applicability. Generally, the research conﬁrms that there is no single model that ﬁts all circumstances; rather, dataset characteristics directly inﬂuence model performance. These results offer real-world guidance on the choice of boosting models and point to the areas where future research, particularly in adaptive and hybrid boosting techniques can be used to further enhance performance and generalization.

AI에게 질문

Bookmark

View Full Paper