BEV-based collaborative distillation and multi-domain transformer fusion for autonomous driving 3D object detection

Introduction Autonomous driving systems in complex traffic environments require accurate detection of 3D objects, such as vehicles, pedestrians, and cyclists. Methods To address challenges such as unstable image depth, sparse Light Detection and Ranging (LiDAR) point clouds, and insufficient cross-modal fusion, this study proposes a Bird’s Eye View (BEV)-based Instance-Context Collaborative Distillation and Multi-domain Transformer Fusion (BICD-MTF) architecture. First, multi-view image features and LiDAR point cloud features are mapped into a unified BEV space. Next, instance structural relationship guided distillation and BEV context response collaborative distillation are employed to enhance cross-modal consistency. Finally, multi-domain collaborative attention fusion and a Transformer decoder are employed to achieve robust 3D object detection. Results Experiments show that the mean average precision and nuScenes Detection Score (NDS) of the complete method reach 69.09% and 72.13%, respectively, which represent improvements of 4.41% and 3.77% over the baseline model. Meanwhile, the model achieves improved localization and motion estimation accuracy, with the mean translation error, mean orientation error, and mean velocity error reduced to 0.284 m, 0.312 rad, and 0.254 m/s, respectively. When the detection distance reaches 100 m, the mean average precision remains at 56.94%, and the NDS maintains at 65.28%. In a continuous 300-frame sequence from the nuScenes validation set, the NDS remains stable between 72.03% and 72.38%. Discussion The results indicate that the proposed method improves the accuracy and stability of 3D object detection under nuScenes-like urban driving scenarios and provides a useful reference for multi-source perception fusion in autonomous driving.