Subject-independent multi-channel voting for EEG-based emotion recognition using wavelet scattering deep network and advanced signal metrics

Abstract Electroencephalography (EEG) signals, reflecting human brain activity, hold potential beyond medical diagnosis, particularly in emotion recognition. Despite the development of machine learning models utilizing EEG data for this purpose, achieving good enough accuracy remains a challenge due to signals complexity and non-stationary nature, especially in extracting effective features that encapsulate temporal and frequency information. This paper introduces a novel hand-crafted feature extraction technique that avoids conventional signal segmentation and analyzes the entire length of EEG signals. This method builds a convolutional network utilizing Wavelet Scattering Transform (WST) blocks, followed by deriving a comprehensive 17-feature set from the raw EEG data and WST scattering coefficients. This integrative set takes advantage of the WST’s ability to produce a signal representation that is stable against noise, invariant to time shifts, and captures both temporal and frequency components while also leveraging the intrinsic properties of the raw data, offering an alternative to the computational deep models. The integration of Linear Discriminant Analysis for dimensionality reduction and the K-Nearest Neighbors algorithm for classification, further refined by a majority voting mechanism across all channels, results in a robust classification framework. The proposed method is evaluated across GAMEEMO and DEAP datasets with two and four emotional classes, using Leave-One-Subject-Out validation, achieving classification accuracy exceeding 97%. The findings support the effectiveness of this approach in EEG-based emotion recognition. Furthermore, an ablation study on the two datasets is implemented to assess each component’s impact, revealing insights into the model’s effectiveness and improvement areas.