Leveraging Federated Self-Supervised Approach for Stress Recognition to Mitigate Label Leakage

Federated learning (FL) enhances privacy in decentralized affective computing by enabling local model training without data centralization. However, FL remains vulnerable to label leakage, compromising privacy in stress recognition tasks. To address this, we propose a federated self-supervised contrastive learning framework, which eliminates the need for labeled data during global model training. By leveraging unlabeled physiological signals, our approach learns generalized representations, significantly reducing privacy risks while maintaining high classification performance. Our method integrates multimodal transformers, effectively modeling physiological signal heterogeneity and cross-modal interactions. Experimental results on real-world stress datasets (WESAD, VERBIO, and DAPPER) demonstrate superior accuracy and robustness, particularly under Non-IID conditions and limited labeled data scenarios. Compared to standard FL approaches, our method achieves better generalization while preventing information leakage, making it highly applicable to privacy-sensitive affective computing applications.