RGB-T pedestrian detection method based on improved illumination-aware weight

Aiming to address the issues of low recognition accuracy, limited all-weather usability, and high missed detection rate in RGB-T multimodal pedestrian detection algorithms, this paper proposes an improved illumination-aware weight-based RGB-T pedestrian detection method. Firstly, the mid-term fusion strategy is adopted by changing the fusion point from after five layers of convolution to after the third layer of convolution. This ensures that the fused feature map contains both detailed and semantic features. Secondly, through learnable parameters, the illumination weights wd and wn are adjusted to more reasonably map illumination information to fusion weights, resulting in new illumination weights wr and wt. Finally, adjustments are made to the network structure of the semantic segmentation module by replacing two-stream semantic segmentation with fused single-stream semantic segmentation using new illumination weights wr and wt. The proposed method's detection ability is validated on a cleaned KAIST dataset. Experimental results demonstrate that compared to the original network model, our proposed method reduces average missed detection rate from 15.69% to 13.15% while improving mAP@0.5 by 4.1%, showcasing superior performance.