Copies in the Benchmark: A Row-Level Audit of the UCR and UEA Time Series Classification Archives

The UCR and UEA archives are the standard benchmarks for time series classification. Comparisons over them assume that test rows are unseen and that datasets are independent. We test both assumptions at the level of individual rows. In the 128 univariate datasets, 23 contain value-identical series, and in 17 of them some test rows also appear in the default training split: 37.2% of test rows in PowerCons, 16.4% in Crop, and 13.0% in Worms. Three of the 30 multivariate datasets also leak on the default split, including 45.7% of test rows in PEMS-SF. In two univariate SmallTrain variants, almost all training rows are rows of the test set: 15 of 17 in InsectEPGSmallTrain and 26 of 28 in FreezerSmallTrain. Over 30 univariate resamples, removing copied rows from the evaluation lowers 1-NN accuracy by up to 12.2 points and MiniRocket accuracy by up to 4.6 points. In published default-split predictions it lowers mean accuracy on Crop from 74.7% to 69.8% across 17 classifiers, but it changes none of 105 significant-or-not outcomes among 15 classifiers. We then treat related datasets as one unit. Merging datasets that share rows, which turns 112 datasets into 93 families, costs less significance than dropping 19 datasets at random, so those datasets add little independent evidence. Families from documented common sources (71 units) hold about as much evidence as 71 independent datasets. HC2, MR-Hydra, and MR keep the top three places, with HC2 first in over 98% of bootstrap samples. Within the top ten, HC2's significant lead over H-InceptionTime does not survive grouping. We release row indices, family maps, and a small audit tool. This is a preprint and has not been peer reviewed. The archive file contains the code, results, and row indices; it holds no archive data. Code is under the MIT License. Project page: https://mahmadmanzoor.github.io/archive-audit/Code and row indices: https://github.com/mahmadmanzoor/archive-auditAuthor: https://mahmadmanzoor.com