UGDMoE: An Uncertainty-Guided Mixture-of-Experts Decoder for Open-Vocabulary Remote Sensing Segmentation

Rapid urbanization and the rapid accumulation of multi-source and multi-temporal Earth observation data are creating an increasing demand for remote sensing models that can flexibly support fine-grained monitoring beyond fixed label taxonomies. Open-vocabulary remote sensing image semantic segmentation (OVRSIS) aims to segment text-specified categories beyond a fixed label space with vision–language foundation models. However, dense remote sensing scenes make pixel–text matching highly vulnerable to semantic confusion and misalignment, owing to extreme scale variation, thin structures, repetitive textures, and prompt sensitivity. To address these challenges, we propose UGDMoE, an uncertainty-guided mixture-of-experts framework for OVRSIS. First, we design a domain-specific MoE decoder with three geometrically specialized experts—for slender structures, mid-scale objects, and large-region context—routed by the alignment-risk cue U0. Second, we introduce a lightweight prompt–response estimation strategy that quantifies prediction dispersion across semantically equivalent prompts to derive U0 in an annotation-free manner. Third, we develop prompt ensemble-based likelihood calibration (PELC), which takes the shared alignment-risk cue U0 as input to calibrate prompt-specific logits before refinement. Finally, we design a lightweight uncertainty-aware structure refinement module that, guided by U0, selectively fuses early visual features with segmentation logits to restore boundary continuity and connectivity of thin structures. We conduct extensive experiments on eight OVRSIS benchmarks under cross-dataset evaluation protocols. Trained on DLRSD, it achieves 46.97 m-mIoU and 63.31 m-mACC, surpassing the strongest baseline by 0.76 and 0.62 points; trained on iSAID, it reaches 37.47 m-mIoU and 58.52 m-mACC, improving over the strongest competitor by 0.71 and 0.61 points. UGDMoE consistently achieves state-of-the-art performance and remains robust under training-source changes.