Reproducibility in machine learning (ML) traditionally relies on shared datasets and fixed train-test splits. However, evaluating Explainable AI (XAI) introduces additional complexity, as explanations depend on both the data and the specific underlying model. In time-series classification, where complex black-box neural networks are increasingly deployed (Fawaz et al. , 2020), the lack of standardized XAI benchmarks (Höllig et al. , 2023) severely hinders progress. Currently, most studies retrain models and recompute explanations from scratch. This not only inflates computational costs but also makes it nearly impossible to disentangle genuine XAI methodological improvements from model variation, preprocessing differences, or the inherent run-to-run variance of perturbation-based post-hoc explainers (Ribeiro et al. , 2018). To ensure reproducibility, we extend standard ML practices by treating datasets, trained models, and explanation pipelines as fixed benchmark artifacts. While mature frameworks like aeon (Middlehurst et al. , 2024) standardize predictive evaluation, and dedicated XAI libraries enable the calculation of explanations, they do not distribute precomputed, resource-intensive XAI outputs. Providing this frozen baseline layer eliminates the computational overhead of recomputing ad hoc explanations, enabling consistent XAI evaluation. Beyond reproducibility, generating sampling-based explanations for time-series is often computationally more expensive than training the models. While traditional ML repositories like OpenML and the UCR Time Series Classification Archive successfully reduce redundant computation, no equivalent initiative stores precomputed post-hoc explanations alongside underlying models for diverse time-series tasks. Existing XAI frameworks, such as Captum (Kokhlikyan et al. , 2020), Quantus (Hedström et al. , 2023), and Alibi (Klaise et al. , 2021), offer valuable tooling but still require resource-intensive model retraining and explanation recomputation. Furthermore, while XAI evaluation datasets exist (Tritscher et al. , 2020), particularly in vision and NLP (Arras et al. , 2022;DeYoung et al. , 2020;Yalcin et al. , 2021;Liu et al. , 2021;Zhang et al. , 2025), they focus on ground-truth annotations rather than reproducible model plus explanation pipelines. Similarly, time-series benchmarks like Exathlon (Jacob et al. , 2021) and XTSC-Bench (Höllig et al. , 2023) do not distribute precomputed post-hoc explanations. Unlike standalone software libraries, ExplainTS acts as a complementary data resource. By providing frozen, ready-to-use explanation artifacts it allows researchers to evaluate new XAI methods and metrics without the massive computational burden of regenerating baselines. Our main contribution is ExplainTS, a comprehensive repository comprising 103 diverse classification tasks. Its core features include: • Shared datasets, frozen train-test splits, and pretrained ConvLSTM-based baselines for consistent evaluation of XAI methods. • Precomputed outputs from multiple model-agnostic explainers (SHAP (Lundberg and Lee, 2017), LIME (Ribeiro et al. , 2016), Anchor (Ribeiro et al. , 2018), and PHAR (Mozolewski et al. , 2026) ), significantly reducing the computational cost of experiments and eliminating run-to-run stochastic variance. • A stable artifact layer (data, models, and explanations) for evaluating advanced explanationquality metrics (e. g. , Lipschitz quotient or Jaccard Index) without re-running expensive pipelines. precomputed artifacts for downstream XAI auditing, lowering the entry barrier for students and researchers. The repository structure (Figure 1) is fully extensible via open-source Python code (github. com/mozo64/papers/tree/main/zenodo-ucr), allowing researchers to seamlessly integrate their own custom metrics or algorithms without re-running expensive baseline pipelines. We actively encourage the research community to contribute novel explainers and evaluation protocols via pull requests, paving the way for collaborative, iterative updates to the Zenodo benchmark. The remainder of this article is organized as follows. Sections 2 and 3 detail the benchmark construction, data characteristics, and intended uses, while Section 3. 4 outlines the Zenodo archive structure. Importantly, comprehensive dataset statistics, exact model performance metrics, and detailed repository layouts are provided exclusively in Supplementary Tables S1-S4. The benchmark covers 103 tasks from the UCR/UEA time-series archives (Dau et al. , 2019;Bagnall et al. , 2018), including 83 univariate and 20 multivariate datasets. We excluded datasets where the standard ConvLSTM architecture failed to converge effectively, ensuring the repository contains only reliable models for subsequent explanation analysis. The selected datasets vary significantly in scale and complexity: total instance counts range from 30 to 24000 (median: 553), time-series lengths span from 8 to 1751 time steps (median: 235), and the number of target classes varies between 2 and 60 (median: 3). While univariate datasets contain a single channel, the multivariate tasks include up to 144 dimensions. Detailed descriptive statistics for each individual dataset are provided in Supplementary Tables S1-S3. Some multivariate tasks (e. g. , Libras, PenDigits) originate from the UCI Machine Learning Repository, but here we rely on the curated UCR/UEA variants of these datasetsfoot₁. No new measurements were collected, and the original licensing terms for all third-party datasets remain with their owners. The raw time-series were downloaded programmatically using the aeon loader (Middlehurst et al. , 2024). Each channel was scaled to zero mean and unit variance with the scikit-learn StandardScaler (Pedregosa et al. , 2011) fitted to the training split and reused in the test split. Each series of length T was segmented into blocks of shape (n steps, n length, F), where n steps is the third-smallest divisor of T above 2 and F is the number of channels. This segmentation yields tensors with a consistent shape, so that the same convolutional architecture can be instantiated and trained separately on each data set, and converts each multichannel time-series into a (steps × length × channels) representation compatible with the ConvLSTM-based implementation. While this dynamic heuristic avoids per-dataset hyperparameter tuning and proved robust across the majority of tasks, it represents a compromise. We did not perform a granular sensitivity analysis for each dataset to quantify potential temporal distortions; however, maintaining a unified architecture was prioritized to establish a stable baseline for XAI evaluation rather than maximizing predictive performance through custom preprocessing. All datasets use a predefined stratified 75/25 train-test Rule Records "prediction": "1", "rule": { "feature₄": [">-0. 74", " -0. 74", " 3. 94", " 1. 89", " <=2. 91", " confidence ": 0. 9565, " coverage ": 0. 7197 } ]All notebooks and helper scripts used to build the ExplainTS benchmark dataset are hosted on a public GitHub repository scripts/ bash utilities for dataset assembly, including compress models. sh and compress explainers. sh (packaging models and individual artifacts), filter move. sh (validating completeness and organizing bundles), compress all. sh (creating the final Zenodo archives), and report. sh (generating coverage statistics). These resources facilitate full reproducibility and provide templates for integrating new explainers. Detailed Zenodo archive structures and their exact contents are documented in Supplementary Material Section 1. 2 and Table S4. 6 https: //github. com/mozo64/papers/tree/main/zenodo-ucr. We characterize the ExplainTS benchmark along two axes: the performance of a shared ConvLSTMbased classifier baseline across datasets and the availability of post-hoc explanations (DeepSHAP, LIME, Anchor) on the train and test splits. While SHAP achieves full coverage and LIME covers all except FaceDetection, Anchor rules Dataset-specific metrics are detailed in Supplementary Tables S1-S3. To practically demonstrate the utility of the benchmark, we provide an educational template, ExplainTS CaseStudy. ipynb. Importantly, this script serves strictly as a technical proof-of-concept Since pretrained models and post-hoc explanations are precomputed for all tasks, the primary
Mozolewski et al. (Tue,) studied this question.