Adaptive semantic fusion based scene aligned segmentation for optimizing the efficacy of multimodal video classification

Multimodal video segmentation and classifications are an emerging need for the detection of video-based fraudulent applications.Existing fusion methods, such as the Multimodal Embedding Alignment System (MEAS), Transformer models, and Visual Semantic Multi-Head Attention (VSMHA), have produced prominent multimodal video classification results.However, they rely on static segmentation and uniform fusion strategies, which weakens cross-modal interaction and leads to the loss of contextual information.Nevertheless, they rely on uniform fusion and static segmentation, which weaken crossmodal interactions and lead to the loss of relevant context.To address these limitations, this paper presents a Dynamic Adaptive Semantic Fusion Framework (DASFF) that integrates an adaptive video segmentation system that synchronises semantic scene changes with segment boundaries in real time.The proposed method enables more coherent and discriminative feature representations by capturing both temporal dynamics and contextual semantics.To demonstrate the efficacy of the proposed DASFF method, an extensive experimental study is conducted using the TAU urban audio-visual dataset.The proposed DASFF achieved an accuracy of 98.70% and demonstrated robust improvements in statistical performance.