Pedestrian Activity Recognition and Prediction With Multi-Dimensional Point Cloud-Based Deep Learning for Millimeter Wave Radar

This study presents a multidimensional point cloud-based pedestrian activity recognition framework for millimeter-wave (mmW) pedestrian collision avoidance sensors. While many existing methods exploit range-velocity-angle data through Fourier-based processing along array or slow-time dimensions, such approaches face fundamental limitations including a high-frequency mmW radar systems, particularly in the 79-GHz band. These include across-range-unit (ARU) range walk due to high range resolution, a limited unambiguous velocity range, and the inherent tradeoff between temporal and velocity resolutions. To address these issues, we propose ak-decomposed weighted kernel density (WKD)-based range-Doppler velocity estimation method that directly extracts the range, Doppler velocity, direction-of-arrival (DOA) angles, and reflection intensity from point cloud data to overcome the shortcomings of short-time Fourier transform (STFT)-based analysis. Furthermore, for pedestrian motion estimation and future trajectory prediction at crosswalks, the proposed framework combines this multidimensional point cloud representation with a Transformer-based model. Specifically, it employs a 12-class classification scheme comprising three walking speed states and four trajectory patterns, and enables accurate prediction of a pedestrian ’s final position (1-2 seconds ahead) using only short-term slow-time observations.