Automated generation of ground truth images of greenhouse-grown plant shoots using a GAN approach

Abstract The generation of a large amount of ground truth data is an essential bottleneck for the application of deep learning-based approaches to plant image analysis. In particular, the generation of accurately labeled images of various plant types at different developmental stages from multiple renderings is a laborious task that substantially extends the time required for AI model development and adaptation to new data. Here, generative adversarial networks (GANs) can potentially offer a solution by enabling widely automated synthesis of realistic images of plant and background structures. In this study, we present a two-stage GAN-based approach to generation of pairs of RGB and binary-segmented images of greenhouse-grown plant shoots. In the first stage, FastGAN is applied to augment original RGB images of greenhouse-grown plants using intensity and texture transformations. The augmented data were then employed as additional test sets for a Pix2Pix model trained on a limited set of 2D RGB images and their corresponding binary ground truth segmentation. This two-step approach was evaluated on unseen images of different greenhouse-grown plants. Our experimental results show that the accuracy of GAN predicted binary segmentation ranges between 0.88 and 0.95 in terms of the Dice coefficient. Among several loss functions tested, Sigmoid Loss enables the most efficient model convergence during the training achieving the highest average Dice Coefficient scores of 0.94 and 0.95 for Arabidopsis and maize images. This underscores the advantages of employing tailored loss functions for the optimization of model performance.