HQSBM: Hybrid Quality-Aware Multi-Fusion-Enabled Deep Learning Model-Based Human Emotion Recognition Using Multimodal Signals

Human Emotion Recognition is defined as the process of identifying the emotions of humans. The traditional human emotion recognition designed using deep learning and machine learning frameworks exhibits certain discrepancies in terms of insufficient data availability, huge computational complexity, overfitting, and inability to solve large-scale data, which has hindered their extensive utilization. Therefore, to alleviate such downsides, an efficient Hybrid Quality-aware multi-fusion-enabled distributed Selective kernel criss-cross attention-based Bidirectional long short-term Memory (HQSBM) is proposed. The inclusion of the Hybrid Quality-aware Multi-Fusion (HQMF) mechanism generates highly correlated features, thereby enhancing the recognition accuracy. The Deep Time Structural (DTS) descriptor extracts the pertinent features effectively and handles high-dimensional inputs faster to enhance the generalizability in emotion recognition. Moreover, the distributed learning mechanism enables the research to handle large, complex signals via the parallel processing ability. Thus, the proposed HQSBM model accurately recognizes human emotions in high-dimensional data, which reduces computational resources effectively. Nevertheless, the results showcase the efficiency of the proposed human emotion recognition research through two datasets, namely the DEAP dataset and the DREAMER dataset. The proposed HQSBM model, when examined on the DEAP dataset, with emotions of anger, surprise, sadness, happiness, and neutrality, delivered an excellent performance. Hence, the experimental results show that the proposed model achieves a high accuracy of 96.57% for 90% training, indicating an improvement of 6.36% relative to the CNN-LSTM model, and 4.49% relative to the CNN-DWT method, representing its efficiency against other techniques.