Integrating Optical and SAR Time Series Images for Unsupervised Domain Adaptive Crop Mapping

Accurate crop mapping is crucial for ensuring food security. Recently, many studies have developed diverse crop mapping models based on deep learning. However, these models generally rely on a large amount of labeled crop samples to investigate the intricate relationship between the crop types of the samples and the corresponding remote sensing features. Moreover, their efficacy is often compromised when applied to other areas owing to the disparities between source and target data. To address this issue, a new multi-modal deep adaptation crop classification network (MDACCN) was proposed in this study. Specifically, MDACCN synergistically exploits time series optical and SAR images using a middle fusion strategy to achieve good classification capacity. Additionally, local maximum mean discrepancy (LMMD) is embedded into the model to measure and decrease domain discrepancies between source and target domains. As a result, a well-trained model in a source domain can still maintain satisfactory accuracy when applied to a target domain. In the training process, MDACCN incorporates the labeled samples from a source domain and unlabeled samples from a target domain. When it comes to the inference process, only unlabeled samples of the target domain are required. To assess the validity of the proposed model, Arkansas State in the United States was chosen as the source domain, and Heilongjiang Province in China was selected as the target domain. Supervised deep learning and traditional machine learning models were chosen as comparison models. The results indicated that the MDACCN achieved inspiring performance in the target domain, surpassing other models with overall accuracy, Kappa, and a macro-averaged F1 score of 0.878, 0.810, and 0.746, respectively. In addition, the crop-type maps produced by the MDACCN exhibited greater consistency with the reference maps. Moreover, the integration of optical and SAR features exhibited a substantial improvement of the model in the target domain compared with using single-modal features. This study indicated the considerable potential of combining multi-modal remote sensing data and an unsupervised domain adaptive approach to provide reliable crop distribution information in areas where labeled samples are missing.