Integrated Bayesian-Bidirectional Attention Network for Advanced Contextual Video Captioning

Automatic video description integrates visual and audio analysis to generate written summaries or captions, crucial for enhancing accessibility and user engagement. However, ensuring accurate and meaningful natural language descriptions remains a primary focus in this field of computer vision. Hence, an Integrated Bayesian-Bidirectional Attention Network (IB-BAN) is introduced for accurate, context-aware, and reliable descriptions of for complex scenes. In previous years, video captioning models often failed to capture the intricate contextual details essential for emphasizing prominent features and actions instead. Thus, a Bayesian Spatial-Temporal Random Fields with RNN is designed to capture and interpret complex spatial and temporal dependencies in video data, which effectively analyzes and understands the intricate relationships and dynamics in video data for video captioning. Multi Graph Adaptive Attention (MGAA) is used to enhance understanding of complex interactions across modalities, enables efficient convergence, and improves the fusion of multimodal data within RNNs, thereby enabling more accurate video caption generation. Furthermore, an Attention-Enhanced Bilinear Correlation RNN (ABC-RNN) is employed to integrate and align multimodal data, such as visual and textual information, to enhance the alignment and fusion of multimodal features, thereby improving the coherence and accuracy of generated captions. These innovations collectively boost the field of video captioning towards more precise and comprehensive descriptions of complex visual content. Overall, the experimental results demonstrate that the proposed model achieves significant improvements of METEOR to 34.6, ROGUE to 52.7, and CIDEr to 52.7 for enhancing linguistic quality and coherence.