Multi-Target Pedestrian Tracking Using Radar and Visual Detection Information Fusion

Pedestrian Multi-object Tracking (MOT) is crucial in surveillance, autonomous driving, and pedestrian density detection. An important challenge in MOT is detecting environmental disturbances like bad weather, occlusion, or tracking failures due to distance. Current methods rely on machine vision, extracting features and establishing trackers. However, these methods solely depend on image data, limiting detection accuracy in unstable conditions. To address this, a method is proposed that combines millimeter wave radar and camera sensors. The radar provides anti-interference, range, and depth measurements, while the camera offers target recognition and orientation. In dealing with the detection relationship between the two sensors, the distance measure in the logarithmic domain is used for matching. In terms of detection results and tracking, the utilization rate of detection results is improved by using the method of secondary matching. This fusion method reduces computational burden and improves stability by complementing missing data from any sensor. The MMW radar and camera fusion tracking algorithm is validated using an experimental platform, demonstrating significant performance enhancements compared to single-sensor tracking.