Answers are grounded in this paper and cite their sources.
Recent healthcare analytics increasingly employ deep learning for patient emotion recognition. However, conventional multimodal methods struggle to effectively capture complementary emotional information across text, video, and signal modalities. To address these limitations, a multimodal patient emotion recognition model integrating ensemble classifiers with weighted optimal score-based fusion is proposed. Initially, text, signal, and video data are acquired through standard resources. Text data are processed via Multihead Trans-Long Short-Term Memory (MHT-LSTM) to generate the first prediction score. Video data are processed using Spatio-Temporal Attention-based EfficientNetB7 (STA-ENetB7) to obtain the second prediction score, while signal data are processed using Spatio-Temporal Attention Autoencoder with LSTM (STA-AE-LSTM) to generate the third prediction score. These scores are integrated using Weighted Optimal-based Fusion of Ensemble Classifiers (WOS-FEC) to obtain the final emotion recognition result. The Refined Random Integer-based Modified Skill Optimization Algorithm (RRI-MSOA) determines optimal weights for individual classifier prediction scores. The proposed model achieves accuracy of 95.515% and lowest False Positive Rate (FPR) of 2.294% in 5th fold across all baseline algorithms and models. Overall, this study develops a multimodal ensemble framework integrating MHT-LSTM, STA-ENetB7, and STA-AE-LSTM, along with RRI-MSOA-based optimal weighting strategy. The WOS-FEC mechanism improves emotion recognition accuracy and reliability, enabling timely analysis in healthcare settings.
No takes yet. Share an insight, caveat, or question.
Bharathi et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: