Synthetic dataset generation for training object detectors for underwater environment

Adapting object detection models to underwater environment presents significant challenges due to underwater light attenuation and limited data availability. This work proposes a novel approach to address the challenge of task-oriented underwater datasets scarcity by generating prompt-guided, realistic synthetic images, and subsequently fine-tuning common object detectors on this data. The proposed pipeline begins by fine-tuning a text-to-image generation model on a small dataset of real underwater images using the Low-Rank Adaptation approach. The model generates a larger set of diverse synthetic underwater images aligned with given textual prompts. To ensure that the generated images are accurate, a verification module is introduced, which filters out misaligned samples based on cosine similarity between prompt and caption embeddings of the generated image. Segmentation masks are automatically extracted from the generated images using a salient instance segmentation model. Finally, we augment real underwater images with these synthetic image-mask pairs and fine-tune a detector model on an extended dataset. Experimental results on a subset of under 800 images from the USIS10K benchmark demonstrate a 2.2 % increase in mean Average Precision (mAP) with a moderate amount of synthetic data compared to the real-only baseline. Although synthetic images are generated for only the human divers and ruins classes, detection improves also for non-augmented classes. We further experimented with a scarcer data setting by reducing the training images of the two target classes from 345 to 180. In this setting, augmenting with our synthetic images showed a larger improvement of 3.3 % in mAP, underlining the usefulness of synthetic data when real training data are limited.