Radar-based Human Activity Recognition Using Multi-Domain Maps and Feature Fusion Convolutional Neural Network

Radar-based human activity recognition (HAR) provides a contactless way for a variety of scenarios such as human-computer interaction, smart security, and advanced surveillance with privacy protection. In this paper, we propose a novel HAR method based on fusion map (FuM) and feature fusion convolutional neural network, which is referred to as FuM-MS-Net. Firstly, the time-Doppler map (TDM), time-range map (TRM) and cadence velocity diagram (CVD) are merged into a fusion map. Secondly, a feature fusion convolutional neural network abbreviated as MS-Net is designed, which is composed of two lightweight networks, MobileNetV3-large and ShuffleNetV2. Thirdly, the fusion map is fed into the MS-Net to realize HAR. Finally, the experimental results based on the frequency-modulated continuous-wave (FMCW) radar public dataset from the University of Glasgow show that the proposed method can achieve the recognition accuracy of 96.88%, which prove the effectiveness of the method.