A novel sub-network level ensemble deep neural network with regularized loss function to improve prediction performance

Abstract In recent years, deep neural networks have been widely used and have achieved great success in various fields such as image recognition, natural language processing, and robotics. One method to increase the accuracy and generalization of neural networks is the use of a group of neural networks, ensemble learning, and aggregation of their outputs. The main idea of this paper is ensemble learning at the layer or block level of the network. This is because each neuron, or each layer, or each block of the network can be considered as a network, and ensemble learning can be performed at the neuron, layer, or block level. In fact, a large neural network composed of a large number of sub-networks (layers or blocks) can be considered as a combination of a large number of small sub-neural networks, and improving the performance of each sub-network can contribute to the overall improvement of the network performance. In other words, a layer of a network or, more generally, a sub-network of a network has the task of extracting or detecting the features of the input data. The more reliable this detection is, the more it can contribute to improving the model's performance. Ensemble learning at the layer or sub-network level can help increase the generalization performance of the layer or sub-network. Better performance of the layers or sub-networks can also lead to better performance of the entire network. The results of testing the proposed method on standard datasets for image classification show an improvement of 0.08 to 7.05 percent in the performance of the proposed model compared to traditional models.