Ensemble and Boosting of Neural Networks and Extreme Gradient Boosting Models for the Modelling of Atmospheric Corrosion of Steel and Zinc

This paper proposes several ensemble and boosting approaches for combining neural networks and extreme gradient boosting (XGB) models for the modelling of atmospheric corrosion of steel and zinc. The invested modelling approaches include bootstrap aggregation of neural networks, bootstrap aggregation of XGB models, bootstrap aggregation of neural networks and XGB models, and further model boosting of the state-of-the-art models using neural networks or XGB. It is shown that bootstrap aggregation of XGB models gives the best prediction performance for the steel corrosion data, while the bootstrap aggregation of XGB models with neural network correction of model errors gives the best prediction performance for the zinc corrosion data in terms of performance on the unseen testing data. Sensitivity analysis is carried out for the top performing models. The sensitivity analysis results show that XGB models tend to give non-smooth sensitivity functions due to the stepwise prediction functions from decision tree-based model. Thus, care needs to be taken when interpreting model sensitivity especially when the data set is small. The bootstrap-aggregated neural networks give smooth sensitivity functions and excellent performance on the unseen data under small data sets.