Enhanced Data Augmentation for Infrared Images with Generative Adversarial Networks Aided byPretrained Models

Abstract Infrared imaging emerges as a promising technique for vision tasks within environments characterized by low or obscured visibility. However, the scarcity of infrared datasets, particularly those comprising paired infrared-visible images, addresses significant challenges for the training of high-performance deep learning models. This paper investigates the application of generative adversarial networks (GANs) for data augmentation in infrared imaging. Our work encompasses a range of GAN models including Pix2Pix, CycleGAN, StyleGAN3, and the proposed Cycle-aided model, which incorporates vision-aided loss to enhance both model training and stability. The results demonstrate that CycleGAN encounters stability issues and is prone to mode collapse, leading to less satisfactory performance. By contrast, the Cycle-aided model that leverages pre-trained models to substantially improve both the discriminative and generative capabilities ofGANs, evidenced by a 51% increase in peak signal-to-noise ratio (PSNR), a 43%increase in structural similarity index (SSIM) and an 18% decrease in the Fr´echetinception distance (FID) over CycleGAN. These improvements underscore the potential of GANs to improve data augmentation practices for infrared imaging in low-visibility environments. The insights gained also pave the way for future research aimed at developing architectures and training strategies of GANS for fully exploiting the unique properties of data augmentation.