Mamba-Based Feature Extraction and Multifrequency Information Fusion for Stereo Matching of High-Resolution Satellite Images

This is the authors' abstract. We don't add key points for this paper.

Disparity estimation is a key technology for recovering 3-D information from satellite imagery. Currently, very high-resolution images pose great challenges for context modeling in disparity estimation. While Transformer-based approaches outperform convolutional neural networks in long-distance modeling, their computational complexity necessitates the reduction of images to smaller patches during initial image processing, leading to substantial loss of contextual information and presenting challenges in ill-posed regions. To address these issues, we propose an end-to-end disparity estimation model, termed Mamba-based feature extraction and multifrequency information fusion for stereo matching of high-resolution satellite images. First, a Mamba-based feature extraction module similar to the U-Net architecture is employed to extract global information from stereo image pairs. Second, to address the challenges of multiscale objects matching and ill-posed regions in satellite imagery, a channel-spatial attention-enhanced multifrequency information fusion module is proposed, which effectively integrates multifrequency information contained in multiscale cost volumes. Finally, in the refinement module, we incorporate gradient data containing high-frequency information and original data with rich intensity information to extract convex weight combinations to obtain the full resolution disparity map, thereby updating a more accurate disparity map with enriched details and refined structures. In the experimental tests, systematic evaluation was conducted by using publicly available datasets, and the proposed algorithm was also compared with traditional and recent stereo matching techniques. Our method ranks first on the WHU-Stereo dataset, significantly outperforming the second-best method by 13.26% inEPEand 20.59% inD1.