What question did this study set out to answer?

The aim is to improve salient object segmentation by integrating depth information with RGB modalities.

January 25, 2026Open Access

Study on salient object segmentation based on depth information guidance and SAM low-rank adaptation fine-tuning

Key Points

The aim is to improve salient object segmentation by integrating depth information with RGB modalities.
Proposed a method using depth information and segment anything model (SAM) for segmentation.
Utilized cross-modal attention to dynamically fuse RGB and depth features.
Employed lightweight low-rank adaptation (LoRA) for fine-tuning while freezing pre-trained weights.
Utilized a UNet decoder to refine segmentation output.
Significant improvements over existing methods in MaxF, MAE, and S-measure metrics.
Superior performance in complex backgrounds and small targets compared to traditional techniques.
Validated through experiments on five benchmark datasets.

Abstract

Accurate segmentation of salient objects is crucial for various computer vision applications including image editing, autonomous driving, and object detection. While research on using depth information (RGB-D) in saliency detection is gaining significant attention, its broad application is limited by dependencies on depth sensors and the challenge of effectively integrating RGB and depth information. To address these issues, we propose an innovative method for salient object segmentation that integrates the Segment Anything Model (SAM), depth information, and cross-modal attention mechanisms. Our approach leverages SAM for robust feature extraction and combines it with a pre-trained depth estimation network to capture geometric information. By dynamically fusing features from RGB and depth modalities through a cross-modal attention mechanism, our method enhances the ability to handle diverse scenes. Additionally, we achieve computational efficiency without compromising precision by employing lightweight LoRA fine-tuning and freezing pre-trained weights. The use of a UNet decoder refines the segmentation output, ensuring the preservation of target boundary details in high-resolution outputs. Experiments conducted on five challenging benchmark datasets validate the effectiveness of our proposed method. Results show significant improvements over existing methods across key evaluation metrics, including MaxF, MAE, and S-measure. Particularly in tasks involving complex backgrounds, small targets, and multiple salient object segmentation, our method demonstrates superior performance and robustness. The significance of this work lies in advancing the application of depth-guided RGB in salient object segmentation while offering new insights into overcoming depth sensor dependency. Furthermore, it opens up novel pathways for the effective fusion of cross-modal information, thereby contributing to the broader development and diversification of related technologies and their applications.

Read Full Paperexternally

KI fragen

Bookmark

View Full Paper