GCMT-Net: Generative Context-Driven Multimodal Temporal Network for Video Anomaly Detection in Consumer Applications

Video constitutes a significant content modality within consumer electronics applications. Video Anomaly Detection (VAD) aims to automatically identify anomalous events deviating from normal patterns in consumer videos. This technology directly supports quality assurance and user experience optimization in consumer electronics products, while also enabling functionalities such as environmental safety monitoring for consumer electronics applications. However, traditional methods rely heavily on single-modal features, making it difficult to effectively utilize the complementary clues between different modalities. They have limitations and semantic gaps in explaining complex scenes and understanding the video context. Moreover, most methods overlook complex cross-modal correlations and fail to fully exploit the potential of multimodal semantic clues. To address these challenges, this work proposes an innovative Generative Context-driven Multimodal Temporal Network (GCMT-Net) to detect abnormal video for consumer electronic applications. The GCMT-Net introduces a context generation network that transforms visual context into rich textual descriptions, alleviating semantic gaps that are difficult to capture with traditional visual features. A novel multi-scale temporal module then integrates visual, auditory, and textual information, significantly enhancing the model’s ability to recognize complex anomalous events and improving its overall robustness. Furthermore, modal consistency learning ensures semantic alignment between modalities and consistent feature representations across scales. Experiments on four benchmark datasets demonstrate the superior performance of the proposed framework in addressing VAD challenges in complex consumer application scenarios.