Adapter‐Enhanced SAR ‐Optical Joint Segmentation With Primary Modality Regularization and Sparse Attention Adjustment

Land cover classification (LCC) through synthetic aperture radar (SAR) and optical data provides comprehensive scene information for more accurate results. However, existing methods have limitations in two aspects: (1) insufficient generalization of foundation vision models in the remote sensing domain, and (2) modal imbalance and ineffective feature fusion in multi‐modal architectures. To address these, we propose APS‐Net with three key innovations: Adapter fine‐tuning strategy, Primary modality regularization and Adaptive sparse attention. Specifically, we present a parameter‐efficient adapter fine‐tuning strategy to integrate a vision foundation model with the LCC task, demonstrating its ability to effectively extract features from both SAR and optical remote sensing images. We introduce a knowledge distillation‐based architecture that leverages a pretrained optical segmentation model to facilitate multi‐modal fusion through employing the primary modality as external regularization. Additionally, we propose an adaptive sparse attention adjustment module that filters out redundant information and suppresses noisy interactions, enabling the effective extraction of relevant SAR and optical features. These components collectively enhance cross‐modal synergy modeling and alleviate modal imbalance and unreliable features. Comprehensive experiments on the WHU‐OPT‐SAR, DDHR and DFC2020 datasets demonstrate optimal performance with APS‐Net outperforming suboptimal method by 1.6% OA and 0.9% Kappa on average.