BrainEmoNet: Emotion recognition network based on brain function asymmetry

The recognition of the subject's emotional states is of great significance for achieving humanized services in many scenarios with human-computer interaction. Recently, identification of the emotional states based on electroencephalogram (EEG) has received increasing attention. However, due to the complexity of EEG signals, EEG-based emotion recognition is very challenging. In this research, a novel BrainEmoNet with learning-based framework is proposed to improve the emotion recognition accuracy from the perspective of the asymmetry of human brain functions. The BrainEmoNet consists of frequency-domain feature network (FFN), long-term dependent feature network (LDFN) and spatial characteristic analysis network (SCAN). The parallel FFN and LDFN are suggested to extract the frequency-domain and long-term dependent features of the information in each brain channel, respectively. Meanwhile, based on the working principle of the human brain, the SCAN with channel-spatial attention mechanism is proposed to focus on the high-value information channels with assigning adaptive weights and analyze the spatial characteristics of the frequency-domain and time-domain features. The feature analysis in the time-frequency-spatial perspective can fully explore the emotional information contained in EEG information. Experimental results on multi-modal DEAP dataset presents the competitive performances of the BrainEmoNet over the existing state-of-the-art models. In the subject-dependent experiments, the proposed model achieves identification accuracies of 86.77% and 82.14% in arousal and valence dimensions, respectively, compared to 75.53% and 72.83% in the subject-independent experiments. The proposed BrainEmoNet model in this research can be used as an auxiliary tool for the assessment or monitoring of emotions.