Rethinking the Key Factors for the Generalization of Remote Sensing Stereo Matching Networks

Stereo matching, a critical step of binocular 3-D reconstruction, has fully shifted to deep learning due to its strong feature representation of remote sensing images. However, the ground truth for the stereo matching relies on expensive airborne light detection and ranging data, thus making it difficult to obtain enough samples for supervised learning. To improve the generalization ability of stereo matching networks on cross-domain data from different optical sensors and scenarios, in this article, we are dedicated to studying the key training factors from three perspectives. 1) When selecting a training dataset, prioritize data with similar regional target distribution as the test set, rather than relying on data from the same sensor. 2) Regarding the training modes, unsupervised methods generalize better than supervised methods. 3) We devised an unsupervised early stop strategy to help preserve the best model based on the pretrained weights. Extensive experiments are conducted to support the previous findings, on the basis of which we present an unsupervised stereo matching network with good generalization performance.