Dynamic Hand Gesture Recognition Using Effective Feature Extraction and Attention Based Deep Neural Network.

Skeleton-based dynamic hand gesture recognition has become the most attractive and popular research domain due to its wide range of applications. Sign language recognition, mixed reality environments, touchless public kiosks, controlling various electronic devices used in daily life, entertainment systems, and more are numerous applications of hand gesture recognition. However, many previous hand gesture recognition methods may fail to achieve satisfactory accuracy due to the lack of spatio-temporal features and noisy skeleton data. In the deployment stage, some hand gesture recognition systems suffer from low execution speed because of the large number of parameters. In this study, we propose a skeleton-based lightweight hand gesture recognition system using a multi-stream feature fusion approach to address these challenges. We extract three different features: y-axis distance, y-axis angle, and global motion features including slow motion and fast motion from the selected 10 joint coordinates. Next, we employ a spatial convolutional neural network (CNN) module to enhance the spatial contextual information for each of the three feature branches. By fusing the three feature branches, robust features are generated and then input into a multi-head attention module, which is subsequently fed into the classification module. Our proposed model achieves high accuracy after evaluation with the DHGD dataset, which demonstrates the superiority of the proposed study.