A Systematic Study on Pretraining Strategies for Low-Label Remote Sensing Image Semantic Segmentation

This paper addresses the critical challenge of semantic segmentation for remote sensing images (RSIs) under extremely limited labeled data. A high-quality initial model is paramount for downstream semi-supervised or weakly supervised learning paradigms, as it mitigates error propagation from the outset. We conducted a systematic investigation into self-supervised pretraining to serve this precise need. Within the low-label regime, we identify and tackle two pivotal factors limiting performance: (1) the domain shift between large-scale pretraining data and specific target tasks, and (2) the deficiency in local feature learning caused by large-window masking in visual foundation model (VFM) pretraining. To resolve these issues, we first benchmark various pretraining strategies, demonstrating that a two-phase General-Purpose Pretraining (GPPT) followed by Domain-Adaptive Pretraining (DAPT) framework is optimal, significantly outperforming both single-phase methods and the existing two-phase paradigm initialized from ImageNet. Subsequently, we propose an Edge-Guided Masked Image Modeling (EGMIM) method for the DAPT phase, which explicitly integrates edge priors to guide the masking and reconstruction process, thereby enhancing the model’s capability to capture fine-grained local structures. Extensive experiments on four RSI benchmarks validate the effectiveness of our approach, showing consistent and substantial gains, particularly in extreme low-label scenarios. Beyond empirical results, we provide in-depth mechanistic analyses to explain the synergistic roles of GPPT and DAPT.