HSM‐MAE: Hilbert‐SSM Memory Autoencoder for Real‐Time Video Anomaly Detection

Video anomaly detection (VAD) remains a challenging task due to the computational burden of modelling long video sequences and the difficulty in capturing subtle anomalies in small spatial regions. In order to address these issues, a novel framework is proposed, namely HSM‐MAE, which integrates scalable spatio‐temporal modelling with discrepancy‐aware memory mechanisms. In the field of temporal modelling, HSM‐MAE employs a state‐space model‐based encoder, facilitating recursive sequence representation with linear computational complexity with respect to sequence length. This property improves scalability when longer temporal contexts are required, rather than guaranteeing an unconditional end‐to‐end speedup under all settings. In order to preserve local spatial continuity, a Hilbert scanning strategy is introduced, which serialises spatial regions within video frames while better preserving their neighbourhood structure, thereby enhancing the model's sensitivity to fine‐grained anomalies. Furthermore, a state memory module is employed to retain latent states across frames, and a state discrepancy metric is designed to measure temporal variation amplitudes, guiding the model to focus on abnormal transitions. A student–teacher reconstruction architecture has been shown to amplify prediction errors in anomalous regions via a distillation strategy, while a random masking mechanism has been demonstrated to strengthen robustness against occlusion and background noise. Extensive experimentation on standard benchmarks demonstrates that HSM‐MAE achieves competitive real‐time throughput and a favourable accuracy–efficiency trade‐off under our evaluation setting, while delivering consistent improvements in detection performance.