A Multi-Scale Deep Multi-Task Network for Integrated Crop Classification, Stress Detection, and Yield Anomaly Estimation Using Sentinel-1 and Sentinel-2 Data

Agricultural monitoring in Sub-Saharan Africa faces three interlinked and operationally critical challenges: crop type identification, stress detection, and yield anomaly estimation. These are typically addressed in isolation, despite their physical interdependence. In semi-arid regions such as Kano State, Nigeria, smallholder rainfed agriculture is especially vulnerable to climate-driven water stress, making integrated monitoring frameworks essential for food security decision-making. Conventional single-task, single-sensor approaches fail to capture the combined effects of phenological diversity, soil moisture dynamics, and precipitation anomalies that govern productivity in these environments. This paper proposes a Multi-Scale Deep Multi-Task Network (MS-DMTN) that performs simultaneous crop classification, stress identification, and yield anomaly estimation through a novel three-stream architecture. The network integrates Sentinel-2 optical data (48 Vegetation Indices), Sentinel-1 SAR data (12 radar features), and hydrological variables (36 climate–soil moisture features) from ERA5-Land and CHIRPS over Kano State. A Gated Attention Fusion Mechanism provides adaptive weighting of all input streams for better multi-task predictions. Stress labels were independently generated from soil moisture and precipitation anomalies, avoiding circular validation. Low agreement between spectral indicators and climate indicators (25.7%) confirms labels’ independence. Using 4734 samples across six crops and four stress levels, MS-DMTN outperformed baseline models and achieved 95.99% classification accuracy, 87.33% stress detection accuracy, and a yield R² of 0.92. Outputs demonstrated ecological consistency, strong yield correlations (r ≈ 0.83), and 90.7% land cover agreement. Results showed that hydrological data significantly improved stress detection and yield prediction, while the attention mechanism improved multi-source integration for scalable agriculture monitoring.