Emotion Recognition Using Short-Term Analysis of Heart Rate Variability and ResNet Architecture

Human emotions serve to have an important role in human behaviors. This study will identify and recognize emotions with the use of facial expressions for the convolutional neural network (CNN) using ResNet architecture and with the use of physiological signals, particularly, heart rate variability and electrocardiogram (ECG). The testing subjects' images were taken, serving as an input to the CNN while the ECG device was attached to them. The emotions of the testing subjects would be the output on the resulting ECG signals and face emotion recognition. These emotions could be happy, sad, neutral, fear, and anger. The data gathered from the testing subjects with the use of the researcher's device worked as intended. The accuracy's obtained from the ECG based on the results was 68.42%.