Enhancing Phytoplankton Recognition Through a Hybrid Dataset and Morphological Description-Driven Prompt Learning

Phytoplankton plays a pivotal role in marine ecosystems and global biogeochemical cycles. Accurate identification and monitoring of phytoplankton are essential for understanding environmental dynamics and climate variations. Despite the significant progress made in automatic phytoplankton identification, current datasets predominantly consist of idealized laboratory images, leading to models that demonstrate persistent limitations in the fine-grained differentiation of phytoplankton species. To achieve high accuracy and transferability for morphologically similar species and diverse ecosystems, we introduce a hybrid dataset by integrating laboratory-based observations with in situ marine environmental data. We evaluate the performance of our dataset on contemporary deep learning models, revealing that CNN-based architectures offer superior stability (85.27% mAcc., 93.76% oAcc.). Multimodal learning facilitates refined phytoplankton recognition through the integration of visual and textual representations, thereby enhancing the model’s semantic comprehension capabilities. We present a fine-tuned visual language model leveraging enhanced textual prompts augmented with expert-annotated morphological descriptions, significantly enhancing visual-semantic alignment and allowing for more accurate and interpretable recognition of closely related species (84.11% mAcc., 94.48% oAcc.). Our research establishes a benchmark dataset that facilitates real-time ecological monitoring and aquatic biodiversity research. Furthermore, it also contributes to the field by enhancing model robustness and transferability to diverse environmental contexts and taxonomically similar species.