Comparison of Different Depth of Convolutional Neural Network Deep and shallow CNN comparison based on FER-2013

The purpose of this paper is to cross-compare the different nature of the decision process of convolutional neural networks (CNNs) of different depths, as well as the characteristics of the ensemble model. The Facial Expression Recognition (FER) problem is selected as the carrier of this experiment. The dataset chosen is FER-2013, one of the most challenging FER datasets due to its complicated and natural contents. This research first trained four models that have different architectures using FER-2013. One is the shallow convolutional neural network, and the other three are deep pre-trained CNNs, including ResNet50 with weights from ImageNet, VGG16 with weights from ImageNet, and VGG16 with weights from VGGFaceNet. Then, by using a gradient-based Class activation mapping technique as a visualization technique, this experiment successfully displayed the different decision-making processes of the selected models and illustrated how the depth of the neural network influences the feature extraction process. This work further experimented with convolutional neural networks(CNNs) having different depths by varying ensemble combinations. Finally, all three-model combinations were tested, and the experiment results show that models that are more different in architecture would result in a better ensemble performance than those with a similar architecture. Thus, one of the inspirations we could get from this work is that models with dramatically different architectures could earn a more remarkable improvement for future ensemble models.