An Extension of Gradient Boosted Decision Tree Incorporating Statistical Tests

Gradient boosted decision tree (GBDT) is one of the most popular machine learning methods for many machine learning and data mining tasks. In this paper, we address the overfitting problem of the existing GBDT algorithms, and introduce the idea of statistical significance into tree construction algorithm to reduce it. We propose a new algorithm, W-GBDT, incorporating Welch's t-test as a tree splitting criteria based on the existing XGBoost algorithm. Our experiment results, using real-world datasets, show that our proposed method significantly outperforms the original XGBoost in both the generalization ability and robustness against the number of iterations.