A multimodal driver monitoring benchmark dataset for driver modeling in assisted driving automation

Abstract In driver monitoring various data types are collected from drivers and used for interpreting, modeling, and predicting driver behavior, and designing interactions. Aim of this contribution is to introduce manD 1.0 , a multimodal dataset that can be used as a benchmark for driver monitoring in the context of automated driving. manD is the short form of human dimension in automated driving. manD 1.0 refers to a dataset that contains data from multiple driver monitoring sensors collected from 50 participants, gender-balanced, aged between 21 to 65 years. They drove through five different driving scenarios in a static driving simulator under controlled laboratory conditions. The automation level (SAE International, Standard J3016) ranged from SAE L0 (no automation, manual) to SAE L3 (conditional automation, temporal). To capture data reflecting various mental and physical states of the subjects, the scenarios encompassed a range of distinct driving events and conditions. manD 1.0 includes environmental data such as traffic and weather conditions, vehicle data like the SAE level and driving parameters, and driver state that covers physiology, body movements, activities, gaze, and facial information, all synchronized. This dataset supports applications like data-driven modeling, prediction of driver reactions, crafting of interaction strategies, and research into motion sickness.