DecoupledFM: Decoupling Invariant and Specific Representations for Robust Cross-Modal Feature Matching

Cross-modal image matching, such as between optical-SAR or RGB-infrared images, remains a critical yet challenging task due to severe, non-linear radiometric distortions that plague existing end-to-end matchers. We identify feature entanglement as a potential cause of this performance degradation: standard backbones implicitly couple modality-invariant geometric structures with modality-specific appearances. This coupling, while effective within a single domain, breaks down under drastic domain shifts, leading to matching failure. To address this, we propose DecoupledFM, a semi-dense matching framework that uses separate invariant and specific feature branches. The core of our method is the Invariance-Guided Disentanglement Module (IGDM), which enforces a trio of synergistic constraints—feature cyclic substitution for modality invariance, reconstruction with an orthogonality loss for disentanglement completeness, and keypoint score prediction for geometric saliency awareness—to achieve a principled decomposition. This novel design effectively injects the structural priors of sparse descriptors into a high-precision semi-dense architecture. During inference, only the disentangled invariant features are used for robust coarse matching via an Efficient Cross-View Matching Module, while fine refinement also uses backbone features. Extensive experiments on 20 challenging subsets demonstrate that DecoupledFM achieves state-of-the-art performance, significantly outperforming existing methods.