Greedy function approximation: A gradient boosting machine.

Function estimation/approximation is viewed from the perspective numerical optimization in function space, rather than parameter space. A is made between stagewise additive expansions and steepest-descent. A general gradient descent “boosting” paradigm is for additive expansions based on any fitting criterion. Specific are presented for least-squares, least absolute deviation, and-M loss functions for regression, and multiclass logistic likelihood for. Special enhancements are derived for the particular case where individual additive components are regression trees, and tools for such “TreeBoost” models are presented. Gradient of regression trees produces competitive, highly robust, interpretable for both regression and classification, especially appropriate for less than clean data. Connections between this approach and the boosting of Freund and Shapire and Friedman, Hastie and Tibshirani are.