CASPEX Image Processing: a tool for space exploration

INTRODUCTIONTo meet the needs of space exploration missions, CNES and its industrial partner 3D+ have developed a series of dedicated cameras called CASPEX1 (Figure-1). These cameras consist of two parts: A detector unit with different numbers of pixels and spectral ranges, as well as filters deposited on it (color, polarization), An optical unit mounted on the detector, with different functional points, such as wide-angle cameras for navigation, or tilted proximity cameras to study the composition of the ground under millimeter resolution. Such cameras are used on Supercam/RMI aboard Perseverance2 and will be used on the MBRSC Rashid35 rovers and on the CNES/DLR Idefix4 rover aboard the JAXA MMX mission. CASPEX Image Processing (CASPIP) is a tool used for the first-level correction processing of the raw images captured by these cameras (Figure-2). CASPIP SOFTWARE DESCRIPTIONCASPIP is an easy-to-use Python library developed at CNES and distributed on https: //www. connectbycnes. fr/en/caspip. It works on Linux and Windows. The configuration parameters are grouped in a camera object, so that it is possible to define several cameras in one run. The calling interface is step-by-step and as simple as follows: processedᵢmage = caspipfunction (mycamera, imageₜoₚrocess, (optional parameters) ) In addition to the main functions shown in Figure-2, some tools are added, such as direct and inverse localization functions and distance measurements, as well as a GUI visualization tool. CASPIP can be used also for enhanced image processing5, such as 3D reconstruction. MAIN FUNCTIONSRadiometric processingCASPEX detectors have a PRNU (Photon Response Non Uniformity) of less than 1. 35%1 and very few defective pixels1. Despite such low levels, CASPIP can use ground data to correct if needed PRNU effects, as well as interpolate missing pixels in the image. The wide-angle cameras induce a strong low-frequency vignetting effect (Figure-6, ab), which can also be observed on various cameras, such as Supercam/RMI (Figure-3, ab). The vignetting is corrected by dividing by a flat-field reference image. Typical color filter arrays provide a sparse representation of the spectral bands using a Bayer pattern, alternating red, blue and green pixels. It is important to perform a demosaicing step, to ensure that the final RGB image has the same resolution as the detectors. Several methods can be used for demosaicing, with the most common being based on interpolation. CASPIP implements the Hamilton-Adams7 (Figure-6, bc) and Malvar8 algorithms by default, although several other techniques can be implemented instead, such as Bayesian approaches6, but with increased computational complexity. Knowing the Modulation Transfer Function (MTF) of the instrument, preferably by in-situ measurements, it is possible to add a deconvolution step to reverse instrument blurring and sharpen the final images. Depending on the camera and the optical design, this step may be necessary (Supercam/RMI Figure-3, bc in this case, the image is oversampled regarding the optical resolution). Before deconvolving, it is necessary to reduce the noise, which should be increased by the processing: CASPIP implements a NL-Bayes denoising technique6 (Figure-3, de). Finally, a colorimetric correction has to be applied for the image to be expressed in a standard camera-independent color space. This is done by applying a correction matrix calculated following the ISO/CIE 11664-3 standard. Geometric correctionsWide-angle cameras cause an important distortion effect that can be corrected by applying a polynomial5 to the (x, y) coordinates (Figure-6, de). This processing causes a deformation of the image, which is no longer on a regular square grid: we can either crop it or keep the whole image (Figure-4), depending on our needs. The resulting image is nominally regularly sampled with a pixel size in the center of the field still equal to the original resolution. Observing the microscopic structure of the ground near the rover requires pointing the camera in a tilted direction, which results in a distorted image because the observed plane is not perpendicular to the pointing direction. This is the case for CAM-M on Rashid-1 and for the MMX wheel cameras, which observe the interaction between the wheels and the regolith. CASPIP allows the calculation of "ortho-images" using the value of the tilt angles (Figure-5). If the ground surface is not flat, residual distortion effects are still expected on the result. FURTHER DEVELOPMENTSSince proximity cameras have a very small depth of field, an object (rock) or a hole will cause defocus to increase in the field of view. We will then develop a function to make the deconvolution variable across the detector field of view. Future CASPEX cameras will also include complex filter arrays9 with typically 8 or 25 spectral bands, which will require a specific "demosaicing" function. We are working on new solutions that will be implemented in CASPIP. Such cameras and processing will allow both a spectral capability and a good spatial resolution without moving parts such as filter wheels. Other developments can be considered in the framework of the collaboration with CNES for new space exploration missions. REFERENCES1 Virmontois, C. et al, CASPEX: Camera for space exploration, in preparation (2024) 2 Wiens, R. C. , Maurice, S. et al, Space Sci. Rev. 217, 4 (2021), https: //doi. org/10. 1007/s11214-020-00777-53 Els, S. G. , Almarzooqi, H. et al, Space Sci. Rev, under submission (2024) 4 Michel, P. et al, Earth, Planets and Space 74 (1), 2 (2022), https: //doi. org/10. 1186/s40623-021-01464-75 Thret, N. et al, Space Sci. Rev, under submission (2024) 6 M. Lebrun et al, IPOL, vol. 3, pp. 1-42 (2013), http: //dx. doi. org/10. 1137/1208749897 Hamilton J. F. Jr. , Adams J. E. Jr. , Patent US5629734A (1996), https: //patents. google. com/patent/US5629734A/en8 Malvar, H. S. et al, International Conference on Acoustics, Speech, and Signal Processing, pp. iii-485 (2004), https: //doi. org/10. 1109/ICASSP. 2004. 13265879 Cucchetti, E. et al, Color and Imaging Conf. , pp205-212 (2022), https: //doi. org/10. 2352/CIC. 2022. 30. 1. 3