The Great Time Series Classification Bake Off: An Experimental Evaluation of Recently Proposed Algorithms. Extended Version

In the last five years there have been a large number of new time series algorithms proposed in the literature. These algorithms have evaluated on subsets of the 47 data sets in the University of California, time series classification archive. The archive has recently been to 85 data sets, over half of which have been donated by researchers the University of East Anglia. Aspects of previous evaluations have made between algorithms difficult. For example, several different languages have been used, experiments involved a single train/test and some used normalised data whilst others did not. The relaunch of the provides a timely opportunity to thoroughly evaluate algorithms on a number of datasets. We have implemented 18 recently proposed algorithms a common Java framework and compared them against two standard benchmark (and each other) by performing 100 resampling experiments on each the 85 datasets. We use these results to test several hypotheses relating to the algorithms are significantly more accurate than the benchmarks and other. Our results indicate that only 9 of these algorithms are more accurate than both benchmarks and that one classifier, the of Transformation Ensembles, is significantly more accurate than all the others. All of our experiments and results are reproducible: we release of our code, results and experimental details and we hope these experiments the basis for more rigorous testing of new algorithms in the future.