Rodar: Robust Gesture Recognition Based on mmWave Radar Under Human Activity Interference

Using mmWave radar to conduct gesture recognition is a promising solution for human-computer interaction. Although many studies have shown initial success, two-fold problems still remain unsolved, namely, the high-strength human activity interference and the difficulty in handling similar gestures. In light of these, we develop a robust mmWave radar based gesture recognition system, Rodar, to achieve accurate recognition of similar gestures under high-strength human activity interference, where a Multi-view De-interference Transformer (MvDeFormer) network is proposed. Specifically, to deal with the strong human activity interference, we design a DeFormer module to capture the useful gesture features by learning different patterns between gestures and interference, thereby reducing the impact of interference. Then, we develop a hierarchical multi-view fusion module to first extract the enhanced features within each view, and effectively fuse them across various views for final recognition. To evaluate the proposed Rodar system, we construct a dataset with seven similar gestures under three common human activity interference scenarios. Experimental results show that the accuracy can achieve up to 93.01%. The code implementations are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Xlab2024/MvDeFormer</uri> .