End-to-end fusion of RGB and low-resolution NIR for enhanced crop-specific NIR generation

Near-infrared (NIR) imaging plays a crucial role in applications such as vegetation monitoring, remote sensing, and environmental analysis. However, NIR imaging systems are often expensive and limited in resolution compared to conventional RGB sensors. To address this limitation, we propose an end-to-end deep learning framework that generates high-resolution (HR) NIR images by fusing high-resolution RGB images with low-resolution (LR) NIR inputs. The framework integrates a lightweight image-to-image (I2I) backbone for extracting rich spatial features from the RGB domain, dual feature extractors operating in a multi-scale paradigm, and a learnable fusion module that adaptively combines spatial and spectral information. A decoder network then reconstructs the fused features into the final HR NIR output. Extensive experiments on paired aerial RGB–NIR crop datasets demonstrate that the proposed method significantly outperforms standalone RGB-to-NIR translation and conventional LR-to-HR NIR super-resolution approaches, improving performance from approximately 27.6 dB to over 34.2 dB Peak Signal-to-Noise Ratio (PSNR) and from 0.90 to 0.95 Structural Similarity Index (SSIM) when fusing 128 × 128 NIR inputs. Notably, even when the NIR input is 32 × lower in resolution (16 × 16), the framework consistently surpasses RGB-only baselines, achieving around 29.6 dB PSNR and 0.91 SSIM. Beyond quantitative evaluation, additional analyses including SSIM error map visualization, cross-resolution and cross-dataset testing, and reconstruction variability assessment provide deeper insight into the robustness, limitations, and behavior of the framework under challenging conditions. The modular design of the architecture further enables flexible adaptation to a broad range of multi-modal and cross-spectral imaging tasks. • New framework fuses HR RGB with LR NIR to generate HR NIR images • Backbone handles translation while attention fuses dual-stream features. • LR NIR improves RGB–NIR translation even at 32× downsampling. • Framework beats RGB-to-NIR and NIR super-resolution by large margins. • Robustness validated via cross-resolution, cross-dataset, and error-map analysis.