Answers are grounded in this paper and cite their sources.
Demonstrates improved segmentation accuracy in remote sensing, indicating enhanced data integration.
Multimodal remote sensing technology significantly enhances the understanding of surface semantics by integrating heterogeneous data such as optical images, Synthetic Aperture Radar (SAR), and Digital Surface Models (DSM). However, in practical applications, the missing of modality data (e.g., optical or DSM) is a common and severe challenge, which leads to performance decline in traditional multimodal fusion models. Existing methods for addressing missing modalities still face limitations, including feature collapse and overly generalized recovered features. To address these issues, we propose STARS ( S hared-specific T ranslation and A lignment for missing-modality R emote S ensing), a robust semantic segmentation framework for incomplete multimodal inputs. STARS is built on two key designs. First, we introduce an asymmetric alignment mechanism with bidirectional translation and stop-gradient, which effectively prevents feature collapse and reduces sensitivity to hyperparameters. Second, we propose a Pixel-level Semantic sampling Alignment (PSA) strategy that combines class-balanced pixel sampling with cross-modality semantic alignment loss, to mitigate alignment failures caused by severe class imbalance and improve minority-class recognition. Experiments on EarthMiss, WHU-OPT-SAR, and ISPRS Potsdam demonstrate that STARS consistently outperforms state-of-the-art methods under missing-modality settings. In particular, STARS achieves 33.04 m I o U (+9.60 vs. Baseline-SAR) on EarthMiss (SAR-only test), and reaches 34.07 m I o U (+5.95 vs. Baseline-SAR) on WHU-OPT-SAR under the same missing-modality evaluation. Additional analyses further verify that STARS establishes reliable cross-modality semantic correspondence while decoupling shared semantics from modality-specific cues, enabling accurate and stable remote sensing semantic segmentation with incomplete multimodal data. The code is released in https://github.com/CVEO/STARS .
No takes yet. Share an insight, caveat, or question.
Wang et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: