TL-PNN: a New Two-stream Lightweight Framework for Pansharpening based on Convolutional Neural Networks

The pivotal role of satellite imagery in our understanding of the world has led to a significant achievement: the capability to capture visuals from space using satellites. However, current optical sensors face challenges in delivering high-resolution images that encompass spatial, spectral, temporal, and radiometric dimensions. In response, the technique of Pansharpening is explored, merging high-spectral-resolution multispectral (MS) images with high-spatial-resolution panchromatic (PAN) images to create an improved spatial and spectral (HRMS) images. Generally, such methods utilize convolutional neural networks trained to implicitly map the MS and PAN inputs, producing a fused product with the highest fidelity. However, despite the remarkable fusion performance compared to traditional approaches, to the best of our knowledge, they suffer from two major issue that make them, to some extent, unsuitable for some remote sensing applications. First, deep learning-based methods generally rely upon a very deep and complex architecture, resulting in a high cost of parameters and computation resources. Second, They are more likely to be tailored to a single specific satellite system, lacking from generalization from unseen satellites. Therefore, in this paper, we introduce a novel global two-stream light weight method for Pansharpening, called TL-PNN, capable of combining data from diverse satellite types across the world. The experimental results show that the approach yields very competitive results on different datasets using the same model, while limiting the complexity of the model.