In complex traffic environments such as low light, strong glare, occlusion and at night, systems that rely solely on visible light single sensors for pedestrian detection have drawbacks such as low detection accuracy and poor robustness. Based on the YOLOv8 convolutional network, this paper adopts a dual-branch structure to process visible light and infrared images simultaneously, fully utilizing feature information at different scales to effectively detect pedestrian targets in complex and changeable environments. To address the issues of insufficient interaction of modal feature information and fixed fusion weights, a cross-modal feature interaction and enhancement mechanism was introduced. A modal-channel interaction block (MCI-Block) was designed, in which residual connection structures and weight interaction were added within the module to achieve feature enhancement and filter out noise information. Introduce a dynamic weighted feature fusion strategy, adaptively adjusting the contribution ratio of different modal features in the fusion process, aiming to enhance the discrimination ability of the key pedestrian area. The training and testing of the network designed in this paper were completed on the visible light and infrared pedestrian detection dataset LLVIP and Kaist. At the same time, the test results of the dual-branch model and the model designed in this paper were further verified in actual traffic scenarios. The results show that the dual-branch YOLOv8 network for visible light and infrared images, which was constructed in this paper, can reliably enhance the detection performance of pedestrian targets in complex traffic environments, including accuracy, recall rate, and mAP@0.5, etc., thereby improving the robustness of pedestrian detection.
He et al. (Thu,) studied this question.