Toward Semantic-Aware Aerial Video Anomaly Detection by Exploiting Multimodal Large Language Model

Drones have become increasingly widely applied in surveillance systems due to their mobility, making aerial video anomaly detection methods more crucial. Anomalies in aerial videos often present as semantic conflicts, such as the presence of unexpected objects or unusual behaviors that do not align with the context. Previous approaches often relied on manually crafted knowledge graphs to detect such conflicts, which suffer from poor scalability. Recently, owing to their sufficient alignment training, multimodal large language models (MLLMs) have emerged as a generalized solution for semantic understanding. However, the direct application of MLLMs does not yield satisfactory anomaly detection performance in aerial videos. First, aerial videos often manifest platform-induced pseudo-motion, which obscures the true motion of objects and exacerbates detection errors. Second, without sufficient labeled data for fine-tuning, generic MLLMs often lack scene-level semantic guidance to reliably distinguish abnormal events that include contextually inappropriate behaviors. To address these challenges, we propose SemAero, an MLLM-based framework to address these challenges by: 1) designing an ego-motion reduction module to enhance model perception on object movement, 2) generating scene-specific prompts adaptively with step-by-step guidance for reasonable output, and 3) refining scores with dual-stream consistent feature for better domain-specific anomaly detection. Evaluated across 8 diverse aerial scenes and 73 sub-datasets, SemAero achieves a 3.09% improvement in AUC-ROC over the second-best model, demonstrating its ability in aerial video anomaly detection.