A CNN and Transformer-Based Framework for Fine-Grained Plant Species Classification in Real-World Environments

Plant recognition plays a vital role in agriculture and biodiversity monitoring, and deep learning, particularly convolutional neural networks (CNNs), has gained increased attention for automating this task. However, CNNs have a limitation in their ability to handle complex patterns due to the difficulty in capturing global contextual information. Furthermore, plant datasets are often created in laboratory environments that minimize discrimination challenges, enabling the analysis of model performance. This study proposes a hybrid deep learning model, HDL-PlantNet, for real-world plant recognition on the primary dataset, the Cyprus Seasonal Flora Image Dataset (CSFID), comprising 27 plant species. The HDL-PlantNet model integrates an EfficientNetV2-S convolutional backbone with a Transformer encoder to capture both spatial contextual and long-range dependencies. Additionally, the Swedish Leaf Dataset is used as a supplementary dataset to analyze the consistency of the HDL-PlantNet under controlled environments. Five benchmark CNN models are used for comparative evaluation, and statistical tests and an ablation study are conducted to assess the results. The proposed model achieved the highest observed Macro-F1 and Macro-AUC scores among the evaluated models, reaching 90.06% and 99.59%, respectively. The results demonstrate that combining convolutional and Transformer architectures yields computationally effective performance in fine-grained plant classification while maintaining a compact model size suitable for further research. This study contributes to real-time plant identification studies and supports informed ecological decision-making.