Enhanced Agricultural Parcel Segmentation Through Multi-Modal Satellite Image Time Series Prediction ​​​​​​​

Land monitoring plays a vital role in modern farming practices and agricultural management. An accurate tracking of status and conditions for agricultural parcels could help farmers and landowners make informed decisions about irrigation, fertilisation, pest control and crop rotation choices. Moreover, it enables early detection of potential problems such as soil erosion, plant disease or invasive species infestations, allowing for timely intervention and mitigation measures. Consequently, this leads to promoting sustainable farming practices, improving crop yields and supporting the long-term health of both farmland and ecosystems. One-off acquisitions of satellite images may not be sufficient to accurately identify land crop categories due to temporal variations in agricultural activities and other natural phenomena. Satellite image time series (SITS) data provide a comprehensive view over time, capturing seasonal changes, crop growth cycles and land dynamics, enabling the development of robust classification models and for a more informed decision-making process in agriculture and land management. In this paper, we tackled the task of semantic segmentation of agricultural fields, using both optical and radar modalities. Our experiments made use of the PASTIS dataset, containing over 2.4k 128 x 128 time series, each acquisition encapsulating 10 relevant Sentinel-2 (S2) bands (out of the 13 bands provided by Sentinel-2, bands B1, B9 and B10 were excluded). We have also experimented with it's multimodal counterpart, namely PASTIS-R, containing corresponding Sentinel-1 (S1) time-series, in both ascending and descending orbit. The versatility of these datasets, acquired over the French metropolitan area, motivates their usage in training algorithms which could be further applied on a variety of environments. While S2-based SITS segmentation has been the most attempted approach in recent years, very few works succesfully address the multimodal fusion of optical and radar data (S1 + S2). In our study, we experimented with both trajectories, aiming to find a good trade-off between accuracy and inference speed. We tested multiple fusion techniques for S1 + S2 prediction, drawing conclusions regarding the best approach, while also proposing a technique to retrieve the most influential/non-influential timestamps, through a cross-attention module encapsulated in a mid-fusion strategy. Our experiments reveal that we can achieve comparable accuracy to state-of-the-art (SOA) methods, albeit with improved computational speed. The development of fast and accurate automatic systems for crop segmentation is mostly driven by the need of rapid climate change adaptation, lowering the risk of future unpredictable losses, while also increasing resilience through the pre-emptive adoption of climate-smart agricultural techniques.