Answers are grounded in this paper and cite their sources.
Text–Video Retrieval (TVR) aims to find relevant videos or text based on cross-modal queries, which has become increasingly important with the rapid growth of online video content. Current CLIP-based methods struggle with diverse and abstract video queries as the image–text similarity is insufficient for complex video–text matching. To address these limitations, we introduce CapVA (Caption as Video Assistant), a framework that leverages segment-aware caption generation for multi-level query alignment. With the generated segment-level and global-level captions, CapVA enhances the text–video understanding hierarchically for both feature representation and similarity calculation: (1) It enriches video representation with generated captions through structured cross-modal interaction via a Hierarchical Interaction Module. (2) It calculates similarity scores using dual branches involving query-video alignment and query-caption matching for global and fine-grained semantic alignment. CapVA also includes a Positive-aware Caption Learning module to dynamically align captions with query contexts and a query expansion strategy to enhance retrieval coverage. Finally, extensive experiments show CapVA’s superior performance over existing TVR methods.
No takes yet. Share an insight, caveat, or question.
Yu et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: