Traffic anomaly detection by fusing spatiotemporal graphs and visual perception

Traffic anomaly detection (TAD) is essential for highway operational safety but remains challenging due to limitations of single-modality visual methods or costly sensor reliance. Current systems exhibit high false positives and missed detections in adverse conditions. To overcome these engineering challenges, we pioneer the integration of spatiotemporal graphs with abnormal object detection (AOD), utilizing dynamic vehicle behavior (e.g., abrupt lane changes, speed variations) to enhance detection robustness in complex highway environments. We propose a comprehensive framework, introducing (i) the visual-and-spatiotemporal traffic anomaly detection (VTAD) task, which uses spatiotemporal trajectory maps as auxiliary cues to improve detection accuracy, (ii) the VTAD-highway dataset, comprising 6 435 video sequences paired with spatiotemporal graphs, carefully curated from real-world highway surveillance footage, and (iii) VTAD-FL, a dual-stage network that combines a linked memory token Turing machine (LMTTM) for spatiotemporal trajectory modeling and a contrastive learning head to optimize feature discriminability. VTAD-FL integrates spatiotemporal trajectory features with visual cues through adaptive multi-scale integration, achieving superior temporal coherence and intra-class compactness. Extensive experiments show that VTAD-FL significantly outperforms existing AOD methods across all metrics, establishing a new practical benchmark for unified visual-and-spatiotemporal traffic anomaly detection in intelligent transportation systems (ITS). The dataset and code are available at https://github.com/hongkai-wei/VTAD .