DA-ViViT: Fatigue detection framework using joint and facial keypoint features with dynamic distributed attention video vision transformer

Abstract Fatigue driving stands as a primary contributor to severe traffic accidents. Despite notable advancements in fatigue monitoring techniques, current methods frequently overlook the utilization of shifts in drivers' body postures induced by fatigue. These postural changes can offer valuable indicators of drivers' levels of alertness and degree of fatigue. Therefore, a real-time system with the ability to process body information is needed. This study presents a novel framework that integrates facial fatigue characteristics with alterations in drivers' upper body postures to comprehensively monitor fatigue across multiple dimensions. The neural network architecture at the core of the proposed approach utilizes a video vision transformer framework, with the self-attention mechanism optimized to accommodate the magnitude of fatigue-induced posture changes. This optimization ensures high accuracy while simultaneously reducing computational expenses. This gives the approach the potential for embedded deployment. According to requirements, the standard DMD dataset captured in real-world driving scenarios has been selected for validating the proposed method. In terms of accuracy, compared to detection methods that solely rely on facial and upper body poses, the multi-feature fatigue detection approach has achieved an improvement of 3.4% and 11.9%, reaching 96.24%. Additionally, it has been observed that the adoption of the Dynamic Distributed Attention-based backbone has led to a significant increase in inference speed of over 30%.