Robust-GBDT: Leveraging Robust Loss for Noisy and Imbalanced Classification with GBDT

Abstract Dealing with label noise in tabular classification tasks poses a challenge in machine learning. This paper introduces Robust-GBDT, a novel approach to enhance the robustness of Gradient Boosting Decision Trees (GBDT) models against label noise and class imbalance in tabular data classification. We present a theoretical insight demonstrating that nonconvex loss functions, under specific constraints, can be effectively employed in GBDT models. Leveraging this foundation, Robust-GBDT integrates advanced GBDT frameworks with robust loss functions, including a newly proposed Robust Focal Loss designed to address class imbalance. Our method overcomes limitations of existing noise-robust boosting techniques, such as binary classification constraints, inability to handle imbalanced datasets and missing values, and computational inefficiency. Experimental results show that Robust-GBDT outperforms existing bagging and robust boosting methods, delivering more accurate predictions and improved generalization capabilities in scenarios with label noise and class imbalance.