A Comparative Study of Different Data Augmentation Techniques for Nuclei Segmentation in Histopathological Images

Deep learning-based methods have achieved significant success in medical image segmentation. However, their effectiveness is often constrained by the limited availability of annotated data. To address this challenge, data augmentation techniques have become a critical strategy to enrich the training set and enhance model generalization. In this study, we systematically evaluate both classical and generative data augmentation strategies for nucleus segmentation on the MoNuSeg dataset. The U-Net architecture was employed as the baseline method. To investigate the effects of augmentation, the classical CutMix approach was applied, along with three generative strategies: Latent Diffusion Models (LDM), MedSegDiff, and SPADE, a GAN-based conditional image synthesis method. Furthermore, we propose a hybrid strategy (CutMix+SPADE), where mixed label masks are used as input to the SPADE generator, producing more diverse synthetic examples. Experimental results demonstrate that all augmentation strategies consistently improve segmentation performance compared to the baseline model. In particular, the proposed CutMix+SPADE method achieved the highest results across all evaluation metrics, including Dice, IoU, Precision, Recall, and F1-score. These findings indicate that combining structural mixing with generative synthesis can significantly enhance model generalization.