SeGANmentation: Segmentation via GAN-Based Image-to-Image Translation

This research explores the potential of state-of-the-art Generative Adversarial Networks (GANs) for segmentation through image-to-image (I2I) translation. The presented model, seGANmentation, combines convolutional neural networks and vision transformers to generate segmentation labels directly from input images. It aims to reduce manual labeling and enhance segmentation performance using synthetic data. Comparative analysis with U-Net as a benchmark in segmentation, shows U-Net achieving an average of 0.99 in accuracy, Dice Coefficient, and Intersection over Union, while seGANmentation scores 0.90, 0.83, and 0.71, respectively. Although it did not outperform U-Net, it highlights the potential for GAN-based segmentation. This work lays the foundation for employing GANs in this novel approach. This paper's proposed framework could be valuable in data-scarce scenarios, enhance task accuracy with synthetic data augmentation, reduce the need for extensive training pairs by refining GAN-generated labels with off-the-shelf segmentors, and achieve unsupervised segmentation, unlike traditional neural networks. These hypotheses remain to be tested in future studies.