Cross-similarity guided contrastive learning for infrared-visible image-to-image translation

Infrared images are widely used in security monitoring and autonomous driving due to their resistance to light changes and adverse weather conditions. But, the low contrast and colorless characteristics of infrared images limit the effectiveness of human observation and subsequent detection and recognition algorithms. Methods for translating infrared images into visible images can overcome the above shortcomings, among which contrastive learning methods using self-similarity features achieve the best performance. However, methods based on contrastive learning face the sub-optimization problem caused by random sampling. And the contrastive loss based on self-similarity features faces the problem of encoding entanglement when used for infrared-visible image translation, that is, the features extracted from different categories of regions cannot be distinguished. Therefore, we propose a cross-similarity guided contrastive learning method for infrared-visible image translation. First, to address the randomness and inefficiency of the contrastive loss random sampling process, a sampling strategy based on information entropy ranking of cross-similarity matrix is proposed to obtain sampling points for subsequent contrastive loss calculation. By calculating the information entropy of cross-similarity matrix between input and generated images and sorting them, the sampling points with the most information can be obtained. Second, to alleviate the encoding entanglement problem of the self-similarity contrastive loss due to the low contrast of infrared images, multi-scale spatially adjacent graph structure consistency loss and spatially separated graph structure consistency loss based on cross-similarity matrices are proposed. Experiments on KAIST and FLIR datasets show that the proposed method has the best score and visual performance compared with multiple advanced infrared-visible translation methods. Ablation experiments further illustrate the effectiveness of the method.