A spatio-temporal anomaly detection system to support understanding of abnormal phenomena in automated manufacturing lines

Automated manufacturing lines perform repetitive cycles, where continuous operation increases the probability of abnormal behavior. However, understanding these abnormalities remains a significant challenge, as expert workers must manually observe and compare normal and abnormal cycles to identify deviations. Motivated by this issue, we propose a video anomaly detection system that identifies the spatial and temporal locations where abnormal operations occur by comparing normal and abnormal cycles in automated manufacturing lines. Detecting anomalies in this context is non-trivial, as lines consist of multiple interacting components, periodic cycles exhibit spatio-temporal variability, and collecting large-scale datasets is often infeasible in real-world settings. Our method extracts spatio-temporal motion blobs to capture individual object activities, followed by an IoU-based sampling strategy to select comparable normal cycles considering the variability. Temporal variations are aligned using Dynamic Time Warping (DTW), enabling the construction of a codebook that efficiently captures typical periodic patterns. This allows us to localize both spatial and temporal anomalies with high precision, even under limited data conditions. Our system was evaluated in two real-world case studies: (1) interference between the chuck and the pallet and (2) delays in the assembly of multiple parts. In both scenarios, it demonstrated more precise localization of spatio-temporal anomalies compared to state-of-the-art methods from both manufacturing and computer science fields. Specifically, our method achieved an ROC score of 0.92 while baselines achieved 0.61 and 0.69 for the temporal abnormal scenario. These results highlight our system's robustness to cycle-level variability and its practical applicability in complex industrial environments.