Abstract Given the increasing incidence of urban traffic accidents, autonomous driving systems must possess advanced pedestrian and vehicle detecting skills. Conventional algorithms perform strongly under favorable conditions but experience substantial degradation in harsh environments characterized by concurrent rain and fog. This paper improves upon the YOLO11 model and proposes a novel spatial adaptive deformation network that overcomes the above limitations and improves detection performance in rainy and foggy conditions through multi-scale feature retention and task dynamic calibration. The network incorporates deformable convolutions into the backbone architecture to replace the original C3k2 module, achieving adaptive adjustment of the receptive field by learning spatial offset parameters. By innovating on the original PAN-FPN structure, a rain-fog focus pyramid that preserves key spatial details is proposed and works in conjunction with an improved omnidirectional convolution kernel network to effectively balance global context and local features. Existing detection heads have limitations in terms of interaction between classification and regression tasks. A task collaboration dynamic modulation head is proposed, which can adjust feature interaction and task weights according to sample distribution. The original loss function was unable to effectively solve the problem of class imbalance. The proposed momentum-adaptive sample correction loss improves vehicle and pedestrian recognition performance by dynamically balancing sample contributions. Extensive studies on the MPID real-world dataset demonstrate that the proposed network outperforms the baseline model, achieving a 43.8% enhancement in frame rate (FPS) and an 16.4% increase in mAP@0.5. The cross-dataset evaluation on the DAWN dataset confirms superior real-world applicability across various environmental circumstances, demonstrating robust generalization with a 3.8% mAP@0.5 improvement. The results demonstrate an effective balance between detection accuracy and real-time performance, markedly improving autonomous vehicle perception in foggy and rainy situations.
Hou et al. (Thu,) studied this question.