Adaptive Multi-Temporal Fusion and Cross-Modal Adversarial Alignment for Robust Driver Fatigue Detection

To address the critical challenges of multi-scale temporal dynamics and sensor-intrusiveness in driver fatigue detection, this paper proposes the Multi-Temporal Fusion Attention Network (MTFA-Net). The framework integrates two core innovations: a Multi-scale Temporal Adaptive Fusion (MTAF) module that dynamically weights short-, mid-, and long-term behavioral features via a scene-aware modulator, and a Physiological–Behavioral Cross-modal Adversarial Alignment (PBCAA) network that implicitly infers latent physiological states (e.g., HRV) from facial videos using adversarial learning and mutual information maximization. Experimental results on RLDD and NTHU-DDD datasets demonstrate that MTFA-Net achieves state-of-the-art accuracy (92.8%) while maintaining high interpretability and real-time efficiency, providing a robust, non-intrusive solution for intelligent cockpit safety.