Explainability and Importance Estimate of TimeSeries Classifier via Embedded Neural Network

Abstract Time Series is sequential data with temporal relationship of causality. Time Series can be univariate, a single ordered data sequence, or multivariate, a collection of more than one ordered data sequences. Multivariate Time Series are complex with inter- and intra-relationship between the ordered data sequences. Time Series is common across disciplines and can benefit from insightful analysis.However, analysis of interpretation and importance estimate within a Time Series context is scarce, especially Neural Network (NN) related approaches.In this work, we demonstrate NN related approaches upon Time Series with the aim of explainability and importance estimate of embedded NN classifers. We conducted an empirical study with univariate and multivariate Time Series, where we compared interpretation and importance estimate from existing embedded NN approaches, an explainable AI (xAI) approach, and our adapted method of Pairwise Importance Estimate Extension from previously. We verified interpretation and importance estiamte via ground truth when it is available, or via a combined approach of Sensitivity Analysis and Reduce and Retrain, where we Retrain with Leave-One-Out and Singleton subsets.