Surveillance video-based traffic accident detection using transformer architecture

Road traffic accidents represent a leading cause of mortality globally, with incidence rates rising due to increasing population, urbanization, and motorization. Rising accident rates raise concerns about traffic surveillance effectiveness. This study aims to develop and evaluate a transformer-based model for accident detection in surveillance video. Traditional computer vision methods for accident detection struggle with limited spatiotemporal understanding and poor cross-domain generalization. Recent developments in transformer architectures have demonstrated superior performance in modeling global spatial and temporal dependencies while enabling parallel computation. Nevertheless, using transformers for encoding both spatial and temporal features greatly increases computational demands, which limits real-time application. To address these challenges, this study adopts a hybrid approach that balances efficiency and performance. However, applying these approaches to automated traffic accident detection is often limited by existing datasets that are small and lack diversity, hindering the development of a robust and generalizable system. To address this gap, we curated a comprehensive and balanced dataset that captures a wide spectrum of traffic environments, accident types, and contextual variations. Utilizing the curated dataset, we propose an accident detection model based on a transformer architecture using pre-extracted spatial video features. The architecture employs convolutional layers to extract local correlations across diverse patterns within a frame, while leveraging transformers to capture temporal dependencies among the extracted features. Moreover, most existing studies neglect the integration of motion cues, which are essential for understanding dynamic scenes, especially during accidents. These approaches typically rely on static features or coarse temporal information. In this study, multiple input strategies are evaluated, including combinations of RGB features and motion cues (optical flow), to determine the most effective representation for accident detection. The proposed model demonstrates strong performance on the curated dataset. The findings indicate that incorporating additional motion cues offers limited gains in classification performance, suggesting that the CNN features integrated with the transformer adequately captured the temporal information needed for the task. The RGB-only approach achieved comparable predictive performance with the lowest latency, approximately 0.533 sec per video, highlighting its computational efficiency and potential suitability for real-time applications. Results were also benchmarked against vision-language models (VLMs), including GPT, Gemini, and Llava-Next-Video, and approaches reported in recent studies to evaluate the proposed method’s effectiveness. The proposed transformer-based architectures with a well-balanced dataset provide an effective solution for traffic accident detection in surveillance video. The results highlight the importance of both temporal modeling and data diversity in building a robust, real-world accident detection system.