Road Traffic Anomaly Detection by Human-Attention-Assisted Text–Vision Learning

With the rapid development of society, the number of road vehicles has increased significantly, leading to a growing severity of traffic accident issues. Timely and accurate detection of road traffic anomalies or accidents is crucial for reducing fatalities and alleviating traffic congestion. Consequently, the detection of road traffic anomalies has become a focal point of research in recent years. With the assistance of computer technologies such as deep learning, researchers have developed more accurate and effective methods for detecting road traffic anomalies. However, the small proportion of anomaly-prone areas in surveillance video frames, combined with the complex and difficult-to-capture patterns of accidents, presents new challenges for the application of deep models to traffic anomaly detection from a surveillance perspective. In light of this, this paper annotates the TADS dataset we previously proposed, a popular text-assisted video representation learning method, to develop a more efficient detection method. Utilizing the well-known video-text model CLIP, we have constructed a detection model that leverages unique text and eye-gaze annotation data from the TADS dataset to learn anomaly representations more effectively, thereby improving the detection of road traffic anomalies from a surveillance perspective. Experimental results demonstrate the superiority of our model for detecting traffic anomalies from a surveillance perspective, as well as the utility of the text and eye-gaze data included in the dataset.