On the road to an open-source co-registration and pansharpening suite for planetary sciences: Developments and case studies

IntroductionEvery planetary exploration study faces a common challenge: integrating images from diverse sources, each one with unique spatial and spectral resolutions, and often, georeferencing inaccuracies. This paper presents the developments of a software suite dedicated to facilitating the integration of surface images of planetary bodies through simplified co-registration procedures and reciprocal resolution enhancement using pansharpening techniques. The study delves into the adopted methodologies, providing a comprehensive performance analysis of the tools. We also demonstrate the practical application of the software suite in real Mars and Mercury data case studies, underscoring its potential in the field. The software will be made available open source for the benefit of the scientific community. Automatic alignmentAccurate ground data alignment is typically adopted for surface studies and is necessary for applications like mosaicking, mapping, and photogrammetry. Despite being a common need, data registration remains a process approached manually: the operator searches for matching features within the images in a Geographic Information System (GIS) and decides the function to align based on experience, trial and error, and software availability. This procedure is time-consuming and greatly influenced by the user's skill and perseverance. Our current research presents an alternative approach using an automatic alignment methodology we developed based on the Scale-invariant feature transform (SIFT) computer vision algorithm 1, a powerful tool inherited from the photogrammetric and computer vision fields. This algorithm can identify significantly more tie points (matching features between images) than a human operator, and in a fraction of the time (Fig. 1). Subsequently, matching points are used in two distinct phases: (i) an initial pre-alignment through a homographic transformation, which allows the general geometry of the images to be matched, and (ii) a refinement phase, which allows local features outside the general topography to be taken into account, operated by software or manually. This procedure considerably reduces co-registration time and permits the achievement of up to pixel-level accuracy across the whole image. PansharpeningBeyond the necessity, integrating different data allows for the mutual improvement of information. In this sense, the pansharpening family of methods represents one of the most discussed and established disciplines in remote sensing. Pansharpening refers to the fusion of the spectral properties of a multi/hyperspectral cube with a higher spatial resolution panchromatic image. In this way, the resulting data will possess high spatial and spectral resolution, gaining the advantages and capabilities of both sources simultaneously. Despite the succession of techniques over more than forty years of development 2, the difficult alignment and the lack of accessible tools have meant that pansharpening has rarely been used in planetary sciences outside the Earth Observation field. Among the few examples, pansharpening has proven appreciable advantages for morphological analysis 3, 4 and image simulation 5. Our developments make it possible to apply different component substitution (CS) pansharpening techniques suitable for different fields of application and purposes. The pixel-wise CS methods are generally considered the most traditional and well-established. Although early applications have been considered to suffer from alteration in the spectral response and its slope, more recent techniques display that this problem is now commonly minimised 6, 7. Validation and case studiesOur suite is not dedicated to a single instrument or condition but is fit for use with images from different sources and with different resolution ranges. A large dataset was adopted for validation, including data from the ExoMars TGO CaSSIS, Mars Reconnaissance Orbiter HiRISE, CTX and CRISM, and Mars Express HRSC instruments on Mars observation, and Messenger MDIS NAC and WAC on Mercury. Furthermore, the chosen images range in lighting conditions, surface geological properties and acquisition geometry, allowing the influence of these factors to be observed (Fig, 2). During testing, spatial resolution increases of up to sixteen times the original were probed, specifically between the colour bands of CaSSIS (4. 5 m/px) and the RED band of HiRISE (0. 25 m/px) (Fig, 3. The results were then analysed using various numerical performance indicators and a visual survey. The evaluation indexes adopted include SAM, RMSE, ERGAS, UQI, PSNR, and SSIM, which are suitable for verifying the resulting images' spectral and structural properties. AcknowledgementsThe study is part of the INAF MiniGrant Combined implementation of CaSSIS and HiRISE data through pansharpening experiments (CUP C93C23008430001) and it has also been supported by the Italian Space Agency (ASI-INAF agreement no. 525 2017- 03-17). References1 Lowe, D. G. (1999). IEEE Int. Conf. Comput. Vis. , 11501157 vol. 2. 2 Vivone, G. , et al. (2021). IEEE Geosci. Remote Sens. Mag. , 9 (1), 5381. 3 Kwan, C. , et al. (2017). IEEE Int. Geosci. Remote Sens. Symp. (IGARSS). 4 Semenzato, A. , et al. (2020). Remote Sens. , 12 (19), 3213. 5 Tornabene, L. L. , et al. (2017). Space Sci. Rev. , 214 (1). 6 Aiazzi, B. , et al. (2007). IEEE Trans. Geosci. Remote Sens. , 45 (10), 32303239. 7 Palubinskas, G. (2013). J. Appl. Remote Sens. , 7 (1), 073526. Fig. 1 Example of CRISM image (on the left) aligned to a reference HRSC image on the right) through matching points between images (linked in green) acquired by SIFT operator. Fig. 2 Example of augmentation of a (a) hyperspectral cube (CRISM hrl00004962₀7ₛr183jₘtr3). The resulting image (b) shows three times the original resolution when combined with HRSC using a MMSE Brovey implementation. Fig. 3 Detail of the maximum resolution increase obtained during testing, from 4. 5 m/px (a) CaSSIS mosaic MY34₀04209₁58₀ RPB) to 0. 25 m/px (b) with PSP₀10394₂025 HiRISE by Gram-Schmidt pansharpening algorithm.