Surgical Video Understanding with Alignment-Preserving Temporal Adaptation and Action Triplet Text Alignment

Surgical workflow understanding requires recognizing procedural phases and fine-grained activities from long-horizon videos, yet acquiring dense annotations for surgical video analysis is costly and requires medical expertise. To address this challenge, we present a text-guided and annotation-efficient framework for surgical video understanding based on a frozen surgical vision–language-pretrained (VLP) encoder and a lightweight temporal adapter. The frozen SurgVLP image encoder provides frame-level visual embeddings, and the temporal adapter aggregates them into clip-level representations while preserving compatibility with the pretrained visual–text embedding space. We evaluate the proposed framework on CholecT50 using text-guided prototype matching for phase recognition and few-shot triplet recognition. Experiments show that temporal adaptation improves phase recognition while preserving the pretrained SurgVLP embedding space. In particular, among the evaluated methods, the proposed Text Contrastive method with rich phase prompts achieves the highest phase recognition performance, outperforming the phase-only baseline. Furthermore, the proposed framework enables classifier-free few-shot triplet recognition in the frozen text space without training a dedicated triplet classifier. These results suggest that effective surgical video understanding under limited annotation depends not only on temporal adaptation but also on preserving alignment with the pretrained text space and using semantically informative text prompts.