Research on cross-modal reconstruction technique of visible and infrared images based on deep learning

At present, affected by image acquisition devices and environmental factors, single-modality images generated have problems such as low resolution, lack of texture information, and shape distortion. Visible light images are rich in information, while infrared images have strong robustness in harsh environments, and the two have strong complementarity. Moreover, traditional cross-modal image reconstruction techniques will lose important image details. With the development of deep learning and the continuous optimization of generative adversarial networks, in order to improve image quality and content information, this paper, based on the cGAN network, realizes the mutual conversion of cross-modal reconstruction features between visible light and infrared images, enabling each to extract complementary information and effectively fuse these information together. In view of the characteristics and connections between visible light and infrared images, the semantic segmentation auxiliary task and the grayscale inversion image auxiliary task of visible light are combined with a two-level cascade network structure to obtain a two-level multi-scale information fusion generative adversarial network(TMIF-GAN). The reconstructed images can not only comprehensively generate a more accurate information structure but also obtain richer detailed texture information. The experimental results show that the cross-modal reconstruction and mutual conversion between visible light images and infrared images have a good effect.