POF Sensor and Thermal Camera Multimodal Fusion for Stress Monitoring

This is the authors' abstract. We don't add key points for this paper.

With the advent of Industry 5.0, increasing attention has been directed toward workers' mental well-being, particularly work-related stress. Given the complexity of dynamic work environments, achieving reliable monitoring while minimizing the impact on workers remains challenging. To address this issue, we propose a multimodal stress monitoring system that integrates a wearable polymer optical fiber sensor with a thermal camera to acquire respiratory, cardiac, and regional facial-temperature information. A graph neural network (GNN) was developed to model modality-specific features and their cross-modal relationships. Multimodal recordings from 20 participants were evaluated under blocked five-fold within-subject and strict leave-one-subject-out (LOSO) protocols. The proposed GNN achieved accuracies of 0.7233, 0.9475, and 0.9225 for within-subject three-class classification, relaxation versus low stress, and relaxation versus high stress, respectively. Under strict LOSO evaluation, the corresponding accuracies were 0.5550, 0.7925, and 0.7375. Compared with conventional machine-learning and temporal deep-learning baselines, the proposed GNN achieved the best performance in five of the six evaluation settings. The experimental and ablation results demonstrate that the proposed framework effectively exploits complementary cardiorespiratory and facial thermal information and supports continuous stress monitoring in complex industrial and healthcare settings.