Multi-spectral Pedestrian Detection via Image Fusion and Deep Neural Networks

The use of multi-spectral imaging has been found to improve the accuracy of deep neural network-based pedestrian detection systems, particularly in challenging night time conditions in which pedestrians are more clearly visible in thermal long-wave infrared bands than in plain RGB. In this article, the authors use the Spectral Edge image fusion method to fuse visible RGB and IR imagery, prior to processing using a neural network-based pedestrian detection system. The use of image fusion permits the use of a standard RGB object detection network without requiring the architectural modifications that are required to handle multi-spectral input. We contrast the performance of networks trained using fused images to those that use plain RGB images and networks that use a multi-spectral input.