Enhanced stress detection via intermediate fusion of intermodal signals

The advent of the digital age and sedentary lifestyles has led to an increase in stress-related health problems, creating a need for wearable systems with sensors for stress monitoring. This paper introduces a multimodal intermediate-fusion method using the WESAD dataset, which combines physiological data captured from chest and wrist wearable sensors using a CNN-based framework. The features are preprocessed using a combination of self-attention, cross-attention and squeeze-and-excite attention mechanisms to model intra- and inter-modality relationships. These features are then combined into a multimodal representation for stress classification. We study the impact of the different attention mechanisms on the model performance and compare their performance. The best performance result was achieved with the combination of cross attention followed by self attention, with a maximum accuracy of 98.1% at optimal hyperparameters.To further evaluate the generalizability of the best-performing model, a rigorous Leave-One-Subject-Out (LOSO) cross-validation was conducted, achieving an overall subject-independent accuracy of 83.56% ± 12.01% across 39 test folds, confirming the model’s ability to generalize to unseen subjects The results demonstrate that the integrated feature representations from different sensors perform better than unimodal sensors. The current research can be extended to build effective stress management systems that can provide personalised stress-reduction strategies by understanding the users’ physiological responses to stressors.