Driver Yawning Monitoring for Connected Mobility Using Spatiotemporal Hybrid Transformer

Driver fatigue is a major cause of road accidents. This paper presents a vision-based system for monitoring driver yawning as an early and reliable indicator of fatigue. Besides generating real-time alerts for the driver, this system can be integrated into connected vehicle technologies to share anonymized risk indicators to nearby vehicles under safety-critical scenarios. The proposed approach is based on a hybrid video transformer (HYVTR) to capture local facial appearance and long-range temporal cues for driver yawning detection. It reduces false positives resulting from other mouth actions including talking and laughing. The paper also proposes a simple labeling protocol based on mouth-aspect-ratio (MAR) frequency thresholding for datasets lacking frame-wise yawning labels. Using the YawDD and NTHU-DDD datasets, the proposed approach achieves accuracies of 93.40% and 99.34%, respectively. It outperforms facial landmarks based spatiotemporal Swin-T and ST-GCN baselines, demonstrating robustness across datasets with challenging pose and illumination variations.