Synthetic versus real: exploring the impact of synthetic data on medical image classification

This study explores the impact of synthetic medical images, created through Stable Diffusion, on neural network training for lung condition classification. Using a hybrid dataset combining real and synthetic images, diverse state-of-the-art vision models were trained. Neural networks effectively learned from synthetic data, its performance is similar or superior to models trained purely on real images as long as the training is carried out under equal conditions: same architecture, same number of epochs, same training style, same resolution of the input image. We selected ConvNext-small as our test architecture. Its best performance when trained with a hybrid dataset (synthetic and real images) was 89% while when trained with purely real images it was 85%. These results were obtained when evaluated with an external validation data set curated by a radiologist. However, hybrid models seem to show a limit in their performance when exploring different training techniques. In contrast, a simpler architecture trained with only real images can take advantage of more complex training regimes to elevate its final performance. In this regard, our best hybrid-trained model (ConvNext-small) achieved an external validation accuracy of 87% while ResNet-34 attained a 93% validation accuracy trained only on real images. Both models were evaluated with the real-image-only dataset provided by the radiologist. This study concludes by comparing our top AI models and radiologists' performance levels.