TAM-Net: Temporal Enhanced Appearance-to-Motion Generative Network for Video Anomaly Detection

Video anomaly detection is a challenging task due to the diversity of anomaly. Existing GAN-based approaches model normal motion pattern through transforming a single image to optical flow map, which tends to learn the mapping between two adjacent frames instead of motion evolution in normal scenes. Therefore, this paper proposes a Temporal enhanced Appearance-to-Motion generative Network (TAM-Net) to model evolution of appearance and motion for normal events. In the motion generative branch, the corresponding optical flow map is generated by a ConvLSTM-based generative adversarial network from consecutive frames to learn normal motion pattern. In order to learn appearance pattern, consecutive frames are reconstructed by a auto-encoder in the reconstruction branch. Temporal encoded features of consecutive frames are shared by these two branches to represent changes of normal appearance along with time. By modeling spatio-temporal evolution of normal events, our network can effectively highlight abnormal regions with high generation errors of the predicted optical flow map and reconstructed frame. Experimental results on three independent datasets, UCSD Ped1, Ped2 and Avenue, demonstrate the competitive performance of the proposed method with the other approaches.