Toward Smart Agriculture: A Novel Environmentally Enriched Multimodal Deep Learning Framework for Olive Peacock Spot Disease Stage Classification and Severity Estimation

Olive cultivation is one of the most economically important agricultural activities in the Mediterranean region, yet its productivity is significantly threatened by olive peacock spot, caused by the fungus Cycloconium oleaginum. This disease reduces photosynthetic activity, induces premature defoliation, deteriorates fruit quality, and causes considerable yield losses. Although deep learning has significantly improved automated plant disease diagnosis, most existing approaches rely solely on leaf images and overlook environmental factors that influence disease development and progression. This study proposes an environmentally enriched multimodal deep-learning framework that integrates RGB images of olive leaves with heterogeneous environmental descriptors, including meteorological conditions, soil characteristics, rainfall-derived moisture indicators, vegetation indices, and environmental stress variables obtained from authoritative public data sources. Visual features are extracted using a fine-tuned ResNet50 convolutional neural network, while environmental descriptors are modeled using a multilayer perceptron (MLP). The extracted features are fused at the feature level to simultaneously perform seven-stage disease classification and continuous estimation of lesion coverage and leaf yellowing within a unified multi-task learning framework. Unlike synchronized field-sensor datasets, the proposed dataset combines publicly available olive leaf images with representative environmental observations, providing a reproducible proof-of-concept for multimodal disease diagnosis. Results demonstrate that incorporating environmental information substantially improves disease-stage recognition, achieving an accuracy of 97.77%, a macro F1-score of 0.9809, and a weighted F1-score of 0.9776. The proposed framework also achieved accurate severity estimation, with a mean MAE of 1.29%, RMSE values of 4.86% and 4.75%, and R2 values of 0.957 and 0.969 for lesion coverage and leaf yellowing, respectively. These findings demonstrate the potential of multimodal deep learning to support precision agriculture and intelligent disease monitoring systems.