Human Activity Recognition with Millimeter-wave Radar Point Clouds Using Feature-fusion based Neural Networks

This study presents a human activity recognition method based on millimeter-wave radar point clouds. A feature-fuse neural network was proposed to combine different features extracted from two subnet. First, millimeter-wave radar point cloud data is preprocessed into different formats, such as 3D voxels, 2D dual-views picture. Second, the voxels and views are fed into 3D and 2D neural networks to extract spatial and temporal information, respectively. The features extracted from the two networks are fused to enhance recognition accuracy. The research introduces innovative preprocessing techniques for both voxels and bi-views data, specifically dynamic denoising and Doppler information embedding, and demonstrated their practicality. The dataset used for the experiments contains 10 different human activity categories. Unlike other studies, it focuses on recognizing both abnormal and normal activities, particularly those that are similar to abnormal ones. This research aims to explore potential scenarios for home health monitoring and achieved a recognition accuracy of 95.30%, showing an improvement of 2% to 7.8% compared to the respective single networks. The feature fusion network model showed an accuracy rate of 95.40% during the testing for detecting abnormal activities. For challenging activity categories, the feature fusion network is able to distinguish between them more easily compared to a single network.