An Improved NeRF-Based Method for Augmenting, Registering, and Fusing Visible and Infrared Images

Multimodal image fusion is an efficient information integration technique, with infrared and visible light image fusion playing a critical role in tasks such as object detection and recognition. However, obtaining images from different modalities with high-precision registration presents challenges, such as high equipment performance requirements and difficulties in spatiotemporal synchronization. This paper proposes an image augmentation and registration method based on an improved NeRF (neural radiance field), capable of generating multimodal augmented images with spatially precise registration for both infrared and visible light scenes, effectively addressing the issue of obtaining high-precision registered multimodal images. Additionally, three image fusion methods—MS-SRIF, PCA-MSIF, and CNN-LPIF—are used to fuse the augmented infrared and visible images. The effects and applicable scenarios of different fusion algorithms are analyzed through multiple indicators, with CNN-LPIF demonstrating superior performance in the fusion of visible and infrared images.