ResoMamba-LLM: multimodal physiological signal stress detection for wearable devices via Mamba and: LLM. reprogramming

This is the authors' abstract. We don't add key points for this paper.

Objective. The accuracy of psychological stress detection hinges upon the precision with which models capture complex and variable physiological responses. Current research primarily employs deep learning techniques to model multimodal physiological responses, aiming to enhance detection performance by leveraging complementary features across modalities. However, existing deep learning approaches face challenges in modeling long-range dependencies and exhibit suboptimal robustness in cross-modal fusion. Approach. To address this, this paper proposes ResoMamba-LLM, a multimodal stress detection framework integrating Mamba and large language models (LLM). Specifically, ResoMamba-LLM employs dual-channel Mamba encoders to process chest and wrist physiological signals respectively, supplemented by a frequency domain branch to capture periodic rhythms. It further utilizes lightweight reprogramming techniques to map physiological features onto the semantic space of the LLM, thereby stimulating its latent contextual reasoning capabilities. Main results. Achieving accuracies of 95.83% and 81.65% on the WESAD and EmoWear datasets, respectively, experimental results conclusively demonstrate ResoMamba-LLM’s robust long-dependency modeling capabilities for multimodal physiological signals and its robust cross-modal fusion abilities. Significance. This provides a novel solution for precise, interpretable psychological stress detection.