Sonar Image Generation by MFA-CycleGAN for Boosting Underwater Object Detection of AUVs

Acquiring large amounts of high-quality real sonar data for object detection of autonomous underwater vehicles (AUVs) is challenging. Synthetic data can be an alternative, but it is hard to generate diverse data using traditional generative models when real data are limited. This study proposes a novel style transfer method, i.e., the multigranular feature alignment cycle-consistent generative adversarial network (CycleGAN), to generate sonar images leveraging remote sensing images, which can alleviate the dependence on real sonar data. Specifically, we add a spatial attention-based feature aggregation module to preserve unique features by attending to instance parts of an image. A pair of cross-domain discriminators are designed to guide generators to produce images that capture sonar styles. We also introduce a novel cycle consistency loss based on the discrete cosine transform of images, which better utilizes features that are evident in the frequency domain. Extensive experimental results show that the generated sonar images have better quality than CycleGAN, with improvements of 15.2% in IS, 56.9% in FID, 42.6% in KID, and 7.6% in learned perceptual image patch similarity, respectively. Moreover, after expanding the real sonar dataset with generated data, the average accuracy of the object detector, e.g., YOLOv6, has increased by more than 48.7%, indicating the effectiveness of the generated sonar data by our method.