What type of study is this?

This is a Systematic Review study.

September 10, 2025

TSB-AutoAD: Towards Automated Solutions for Time-Series Anomaly Detection

Key Points

Over half of existing time-series anomaly detection solutions do not outperform random choice, revealing significant gaps.
The study introduces TSB-AutoAD, a benchmark combining 20 methods and 70 variants for comprehensive performance evaluation.
Naive ensembling achieves high accuracy but incurs substantial computational overhead, limiting practical applications.
Selective ensembling balances retrieval efficiency and accuracy by integrating model selection and ensembling strategies.

Abstract

Despite decades of research on time-series anomaly detection, the effectiveness of existing anomaly detectors remains constrained to specific domains - a model that performs well on one dataset may fail on another. Consequently, developing automated solutions for anomaly detection remains a pressing challenge. However, the AutoML community has predominantly focused on supervised learning solutions, which are impractical for anomaly detection due to the lack of labeled data and the absence of a well-defined objective function for model evaluation. While recent studies have evaluated standalone anomaly detectors, no study has ever evaluated automated solutions for selecting or generating scores in an automated manner. In this study, we (i) provide a systematic review and taxonomy of automated solutions for time-series anomaly detection, categorizing them into selection, ensembling, and generation methods; (ii) introduce TSB-AutoAD, a comprehensive benchmark encompassing 20 standalone methods and 70 variants; and (iii) conduct the most extensive evaluation in this area to date. Our benchmark includes state-of-the-art methods across all three categories, evaluated on TSB-AD, a recently curated heterogeneous testbed from nine domains. Our findings reveal a significant gap, where over half of the existing solutions do not statistically outperform a simple random choice. Foundation models that claim to offer generalized, one-size-fits-all solutions have yet to deliver on this promise. While naive ensembling achieves high accuracy, it comes at a substantial computational overhead. Conversely, methods leveraging historical datasets enable fast inference but suffer under out-of-distribution conditions. To address this trade-off, we propose a selective ensembling solution, which combines model selection with ensembling to offer a lightweight, practical balance between accuracy and efficiency. We open-source TSB-AutoAD and highlight the need for more robust and efficient solutions.

Mark Helpful

Bookmark

Relay