In-vehicle hand activity recognition using integration of regions

In this paper, we focus on the analysis of naturalistic driver behavior using hand activity. To that end, a dataset of color and depth images under varying operating modes and illumination settings was collected. The proposed framework provides a robust solution for localizing the hands by partitioning visible and depth images into disjoint sub-regions which may be of interest for studying the state of the driver: wheel, lap, hand rest, gear, and infotainment region. Different feature extraction methods are proposed and thoroughly studied in terms of speed and performance for each of the five regions. A model for hand presence is learned for each region separately, and these are integrated using a second-stage classifier. As the appearance of hands varies among regions and the hands can only be found in a subset of the regions chosen, the technique leverages information and confidence from multiple regions to produce hand activity classification.