AI-Driven Federated Unsupervised Video Summarization with Ensemble Reinforcement Learning and Explainable Caption Generation

Video summarization is realized as an automated process of reducing long videos into compact representations that effectively describe their content and significance. Video summarization techniques are limited by their high computational resource needs, lack of ability to model long-term dependencies properly, redundancy from frame-to-frame representation, and lack of interpretability. To address these challenges, this paper proposes a novel unsupervised video summarization framework, BiSpiDRL_DOA, that integrates interpolation, hybrid transformer-based temporal modeling, optimized keyframe selection, and explainable caption generation. Initially, video frames are extracted and enhanced through Kolmogorov-Arnold Neural Network (KANN)-based interpolation and multi-feature fusion. A hybrid Vision Transformer-Swin Transformer (ViT_Swin) model is then employed to estimate frame-level importance scores. Keyframes are selected using a Binarized Spiking Deep Reinforcement Learning (BiSpiDRL) model optimized by the Dream Optimization Algorithm (DOA) to reduce redundancy and improve selection accuracy. Furthermore, a BLIP-based caption generation module combined with Grad-CAM produces interpretable textual summaries in Kannada. To support privacy-preserving deployment, Federated Learning (FL) is incorporated for distributed model training without sharing raw video data. Experimental results demonstrate the effectiveness of the proposed framework, achieving a 0.98 success rate, 0.97 sensitivity, 0.96 PPV, and 0.96 F1-score, indicating its suitability for efficient and interpretable real-time video summarization.