From Disease Recognition to Agronomic Action: Vision-Language Reinforcement Learning for Explainable Crop Monitoring Using PlantVillageVQA

Timely crop disease diagnosis matters most when it leads to a useful action. Many plant-disease systems still stop at image classification, leaving farmers and extension officers to translate a model label into a field decision. This study introduces a vision-language reinforcement learning framework for explainable crop monitoring and action recommendation using PlantVillageVQA. The framework combines a frozen contrastive language-image pre-training (CLIP) visual encoder with a CLIP text encoder applied to structured question-answer descriptions, producing a 1,024-dimensional state representation for offline policy learning. A Double Deep Q-Network is trained to select one of four agronomic actions: monitor, treat, isolate, or refer to an expert. On a stratified sample of 1,989 images covering 17 disease classes, the proposed policy reaches 94.4% accuracy and 0.861 macro-F1-score on the four-action recommendation task. Compared with image-only logistic regression, it improves accuracy by 19.3 percentage points; however, a fusion-based support vector machine remains slightly stronger as a static discriminator, confirming that the main contribution of vision-language reinforcement learning is not only classification accuracy but the conversion of multimodal evidence into an interpretable decision policy. Ablation analysis shows that causal-reasoning and comprehensive-description question types contribute most to decision quality, while short general-health answers add little beyond images. Shapley additive exPlanations analysis and rule-grounded natural-language templates provide action-level explanations suitable for non-specialist users. The results support the value of multimodal, evidence-to-action AI for digital agriculture in resource-constrained settings, while also demonstrating that field validation, locally adapted rewards, and lighter deployment models are necessary before operational use.