Dual-Task Human Activity Sensing for Pose Reconstruction and Action Recognition Using 4-D Imaging Radar

Radar-based human activity sensing possesses the advantage of penetrating detection, making it highly promising for applications in the fields of security, rescue, and medical treatment. With the advancement of deep learning technology, there is a growing interest in the field of human activity sensing with radars, including tasks such as action recognition and pose reconstruction. However, existing research usually treats these two problems as separate tasks. In this work, we propose a dual-task framework for jointly reconstructing human 3-D pose and classifying human action from 4-D radar images. Our proposed framework uses an ultra wideband multi-input multi-output (MIMO) radar as the detection sensor to obtain the range–azimuth–height–time 4-D imaging data of human targets. A human pose reconstruction network based on 3-D convolutional neural network (CNN) is then used to reconstruct the 3-D human pose, while a dual-branch network based on multiframe 3-D human poses and 4-D radar image is used to classify the human action. To evaluate the performance of the proposed framework, a dual-task dataset is constructed by merging 4-D imaging radar and camera data. Experiments and multiscenario measurements are constructed to validate the effectiveness of the proposed dual-task framework. The results demonstrate that the proposed dual-task network significantly improves the accuracy of both tasks, while providing a single solution for human action recognition and pose reconstruction.