Stereo based gesture recognition invariant to 3D pose and lighting

This paper describes a stereo-based approach for gesture recognition that works well under extreme lighting conditions and tolerates a large range of hand poses. The approach proposes a hybrid gesture representation that models the user's arm as a 3D line and uses images to represent the hand gestures. The algorithm finds the arm orientation and the hand location from the disparity data and uses this information to initialize a color based segmentation algorithm that cleanly separates the hand from the background. Finally, our approach uses the arm orientation to compute a frontal view of the hand through perspective unwarping producing easily recognizable hand gesture templates. The classification algorithm uses statistical moments of the binarized gesture templates to find the match. We achieve 96% recognition rates under varying lighting and 3D poses for a set of 6 gestures.