Probabilistic memory auto-encoding network for abnormal behavior detection in surveillance video

Abnormal behavior detection in surveillance video, as one of the essential functions in the intelligent surveillance system, plays a vital role in anti-terrorism, maintaining stability, and ensuring social security. Aiming at the problem of extremely imbalance between normal behavior data and abnormal behavior data, the probabilistic memory model-based network is designed to learn from the distribution of normal behaviors and guide the detection of abnormal behavior. An auto-encoding model is employed as the backbone network, and the gap between the predicted future frame and the real frame is used to measure the degree of abnormality. An autoregressive conditional probability estimation model and a normal distribution memory model are employed as auxiliary modules, to achieve the prediction of normal frames. When extracting temporal and spatial features in the backbone network, the causal three-dimensional convolution and time-dimension shared fully connected layers are used to avoid future information leakage and ensure the timing of information. In addition, from the perspective of probability entropy and behavioral modality diversity, autoregressive probability model is proposed to fit the distribution of input normal frame, so the network converges to the low entropy state of the normal behavior distribution. The memory module stores the feature of normal behavior in historical data, and injects the current input data. The memory vector and the encoding vector are concatenated along the time dimension and input to the decoder, realizing normal frame prediction. Using public datasets, ablation and comparison experiments show that the proposed algorithm has significant advantages in anomaly detection.