Dual-bone dynamic time–frequency masking transformer-based framework for emotion recognition from functional near-infrared spectroscopy signals

Emotion recognition based on functional near-infrared spectroscopy (fNIRS) has shown great potential for decoding neurophysiological responses using artificial intelligence (AI). However, existing learning methods suffer from limitations in modeling coupled time–frequency dynamics, adapting to individual variability, and capturing long-range dependencies in hemodynamic responses. We proposed a new AI architecture named dual-bone dynamic time–frequency masking Transformer-based framework (DTFMformer), to independently capture transient temporal fluctuations and frequency-specific neurophysiological patterns for robust multi-class emotion recognition. Specifically, the dynamic temporal masking module leverages the coefficient of variation to adaptively identify temporally unstable high-variability patterns and performs masked representation learning. Meanwhile, the frequency masking bone integrates hemodynamic prior information and a dynamic frequency interpolation mechanism. These dual-domain features are deeply characterized through a unified Transformer decoder to capture long-range dependencies and fused through a multi-head cross-attention mechanism. Behavioral validation using the Profile of Mood States showed significant differences between induced states ( p < 0.001 ). Moreover, hemodynamic analyzes identified distinct emotion-specific activation patterns in the prefrontal and temporal cortices, further supporting the validity of the induced states. Experimental results on the publicly available NEMO dataset and two self-collected datasets demonstrate that DTFMformer achieves excellent classification performance over baseline methods. This deep learning approach provides a robust framework for AI-driven neurophysiological signal analysis in affective computing.