Employment and Investigation of Various CNN Models and Datasets for Facial Expression Recognition and Classification

Facial expression recognition (FER) has become a popular research topic serving enhancement of human-computer interaction. Facial expressions are valuable sources of information, especially for identifying emotions. Therefore, classification of emotions is crucial for computers to interpret our reactions. Machine learning algorithms can be employed for solving facial expression classification problems. Among the deep learning algorithms, the convolutional neural network (CNN) architectures are known to be especially suitable for such tasks. In this study, we trained CNN architectures of VGG16, VGG19, and ResNet50 by employing FER2013, FER2013plus datasets, and a subset of AffectNet dataset composed of images of facial expressions. We have used transfer learning to incorporate network models previously trained via ImageNet dataset. We have investigated the impact of various datasets, network models, and parameters on the performance of a multi-class facial emotion classification problem. Due to dataset imbalances, certain data augmentation techniques have also been utilized to increase accuracy.