Combining contrastive frame selector and context-attentive adversarial networks for unsupervised video summarization

Video summarization aims to automatically select the most informative frames to concisely represent video content, but existing approaches face two critical challenges: (1) the insufficient ability to capture the diversity and representativeness of key frames, and (2) the limited capacity to model long-range temporal dependencies within video sequences. To address these challenges, we present an innovative framework for unsupervised video summarization, termed Contrastive Learning and Generative Adversarial Network for Video Summarization (CL-GAN), which integrates contrastive learning and generative adversarial networks to enhance summarization performance. Specifically, the proposed Contrastive Frame Selector (CFS) leverages contrastive learning to distinguish key frames from non-key frames, ensuring that the selected frames are highly representative of the video’s content. By embedding video frames into a feature space, CFS encourages the model to pull similar frames closer and push dissimilar frames apart, improving the overall quality of the selected frames. In addition, we propose a Multi-head Attention Encoder-Decoder (MAED) to replace the traditional attention mechanism, enabling the model to capture more comprehensive contextual relationships across video frames. By combining these components within a generative adversarial network, CL-GAN generates diverse, representative, and temporally coherent video summaries. Extensive experiments conducted on the TVSum and SumMe benchmark datasets demonstrate the effectiveness of the proposed CL-GAN framework, achieving F-scores of 61.1% and 51.5%, respectively. Compared with representative unsupervised video summarization methods, the proposed framework achieves performance improvements of up to 0.8% and 1.1%, respectively.