Temporally transferable crop mapping with temporal encoding and deep learning augmentations

Detailed maps on the spatial and temporal distribution of crops are key for a better understanding of agricultural practices and for food security management. Multi-temporal remote sensing data and deep learning (DL) have been extensively studied for deriving accurate crop maps. However, strategies to solve the problem of transferring crop classification models over time, e.g., training the model with data for a recent year and mapping back to the past, have not been fully explored. This is due to the lack of a generalized method for aggregating optical data with regard to the irregularity in annual clear sky observations and the scarcity of multi-annual crop reference data to support a more generalized DL model. In this study, we tackled these challenges by introducing a method namely Temporal Encoding (TE) to capture the irregular phenological information. Subsequently, we adapted and integrated two methods, i.e., Random Observations Selection (ROS) and Random Day Shifting (RDS) to simulate the variability of temporal sparsity as well as the shifts of crop phenology over different years. We tested this approach with a 1-dimensional Convolutional Neural Network (1D-CNN) and a Transformer Network models. Our results for both classifiers showed that models trained with crop reference data from 2018 and a dense time series of Landsat 7/8 and Sentinel-2 A/B data can be transferred with little decreases in accuracy to map 12 consecutive years from 2010 to 2021. The Transformer Network was slightly more accurate, while the 1D-CNN was much three times faster. Furthermore, the proposed models could achieve similar performances in the same years with and without fully available satellite information. The TE with ROS and RDS appears well suited for improving temporal transferability to support long term historic crop mapping.