A Multitier Stacked Ensemble Algorithm for Improving Classification Accuracy

For real-world problems, ensemble learning performs better than the individual classifiers. This is true for datasets that have many instances closer to the decision boundary. Using a meta-learner to learn from the predictions of the base classifiers generalizes better. Hence, stacked ensemble (SE) is preferred over other ensemble methods. We extend the SE and propose a multitier stacked ensemble (MTSE) algorithm with three tiers, namely, a base tier, an ensemble tier, and a generalization tier. The base tier uses the traditional classifiers to predict the labels. Tenfold cross-validation is used to validate the models in the base tiers. The cross-validated predictions are combined using combination schemes in the next tier. The predictions from the ensemble tier are generalized using meta-learning to give the final prediction. When tested with 36 datasets, the MTSE gives superior performance over the SE. It achieves high accuracy and does not suffer from over-fitting/under-fitting.