Adaptation of deep learning models to distinguish species with strongly overlapping characteristics: the example of spores of the genus Riccia

The liverwort genus Riccia is fairly characteristic in itself, but differentiation between some of the species herein proved as complex. Morphological thallus characteristics show a wide overlap and variation, also influenced by environmental conditions, whereas spore characteristics might be more consistent within individual species. Thus, here we investigated if morphological spore characteristics can be analyzed by means of generative as well as discriminative deep learning models, allowing a differentiation between very similar species. We applied a modified Generative Adversarial Network on spore images of the genus Riccia to generate images, that have a better expression of class specific morphological features. We also created one single spore image for every species that comprises the representative species characteristics. This approach allowed us to also distinguish morphologically very similar taxa and to quantify the morphological similarity between them. Our results show that generative modeling can improve morphological species classification in biology and provides a framework to quantify qualitative species features and similarities. These new tools facilitate more accurate and efficient species identification, thus greatly advancing methodologies in taxonomic research and environmental monitoring.