Driver assistance for Keeping Hands on the Wheel and Eyes on the Road

Motivated from a basic tip for safe driving, "Keeping Hands on the Wheel and Eyes on the Road", this paper introduces a vision based system for driver activity analysis by observing 3D movement of driver's head and hands from multiview video. From the results of upper body and head pose tracking, semantic descriptions of driver activities are extracted in two steps: First, we determine basic activities of each upper body part (e.g. Two, one, or no hand is on the steering wheel; Head is looking left, straight, or right). Then these basic activities are combined in a fusion step to extract higher level of semantic description of driver activities (e.g. whether the driver is following the above safety tip or not). Our experimental evaluation with real-world street driving shows the promise of applying the proposed system for both post analysis of captured driving data as well as for real-time driver assistance.