Single Image Depth Estimation Based on CLIP Language Prior Injection

Single-image depth estimation models perform well on data within their training distributions but exhibit insufficient generalization and robustness when confronted with out-of-distribution data or complex scenes. Existing approaches are largely confined within the visual modality, lacking high-level semantic understanding and struggling with challenges like occlusion and texture loss. To address this, we introduce the Contrastive Language and Image Pre-training (CLIP) model, which achieves strong open-world generalization capabilities through contrastive learning on 200 million text-image pairs. However, a gap exists between CLIP’s scene-level representations and the pixel-level geometric regression required for monocular depth estimation. To bridge this gap, we design a Text-guided Reverse Cross-Attention (TRCA) module to simulate the relationship between text and pixels, enabling fine-grained semantic enhancement from text to pixels. Concurrently, we propose a Text-Guided Depth Bins Center Predict (TBCP) module that utilizes gated attention to filter scene structural semantic features, transforming continuous depth regression into semantically guided discrete distribution estimation. Experimental results on the NYU Depth v2 dataset show that our method achieves a 3.5\% reduction in absolute relative error (Abs Rel) and a 5\% reduction in root mean square error (RMSE) compared to PixelFormer. Furthermore, evaluations across four distinct test datasets confirm that our model exhibits superior generalization capability and enhanced robustness for depth estimation in complex scenes.