The MVTec AD 2 Dataset: Advanced Scenarios for Unsupervised Anomaly Detection

In recent years, performance on existing anomaly detection benchmarks like MVTecAD and VisA has started to saturate in terms of segmentation AU-PRO, with state-of-the-art models often competing in the range of less than one percentage point. This lack of discriminatory power prevents a meaningful comparison of models and thus hinders progress of the field, especially when considering the inherent stochastic nature of machine learning results. We present the MVTecAD2 dataset, a collection of advanced anomaly detection scenarios with more than 8000 high-resolution images from eight object categories. It comprises challenging and highly relevant industrial inspection use cases that have not been considered in previous datasets, including transparent and overlapping objects, dark-field and backlight illumination, objects with high variance in the normal data, and extremely small defects. We provide comprehensive evaluations of state-of-the-art methods and show that their performance remains below 60% average AU-PRO. Additionally, our dataset provides test scenarios with lighting condition changes to assess the robustness of methods under real-world distribution shifts. We host a publicly accessible evaluation server that holds the pixel-precise ground truth of the test set ( https://benchmark.mvtec.com ). All image data is available at https://www.mvtec.com/company/research/datasets/mvtec-ad-2 .