Online Multi-Object Tracking from A Bird's-Eye View by Fusion of Millimeter-Wave Radar and Vision

Multi-Object Tracking (MOT) is an important approach to perceive the surrounding environment and plays the key role in ensuring driving safety in the field of autonomous driving. However, existing MOT algorithms are facing challenges arising from the underutilization of sensor information, the neglect of sensor reliability, and the inability to effectively handle lost trajectories. This paper proposes a multi-sensor fusion algorithm based on millimeter-wave radar and visual sensor for MOT, which utilizes the average height of objects to back-project the image bounding boxes onto the bird's eye view plane. Appearance features and motion orientations are then used to expand the similarity measure between targets, adopting a multi-level data association method based on object fusion states to improve association accuracy. This also utilizes occlusion prediction and recovery methods to recover lost trajectories. For the performance evaluation, an urban environment dataset containing twenty sequences is collected through extensive experiments. The experimental results demonstrate that the proposed algorithm outperforms both the single-sensor tracking algorithm and traditional multi-sensor fusion tracking algorithm, in terms of tracking accuracy and robustness.