Causalformer: Causal Discovery-based Transformer for Multivariate Time Series Forecasting

The forecasting of time series is finding increasingly widespread applications in real-world scenarios, such as medical data and electricity consumption. Recent work primarily employs the Transformer and its variant to capture broad temporal dependencies from time series. However, most approaches often overly focus on the impact of temporal relation, while neglecting the interactions among multivariate elements, and lacking the application of causal relationships. Int the paper, we introduce a Transformer variant based on causal discovery to discover and utilize causal relationships, termed the Causalformer. Our Causalformer not only captures temporal features but also learns a Granger causality graph from multivariate time series, resulting in a sparse adjacency matrix that represents the causal relationships between variables. This allows us to leverage causal relationships to estimate the degree of influence among multivariate elements, thereby further enhancing forecast accuracy. Building upon this, we configure the decoder part as multi-headed output to utilize the causal matrix for understanding how each variable is influenced by others. To accelerate the model’s learning speed, we also design the decoder as generative, allowing it to output an extended time series sequence in a single shot rather than step-by-step, significantly improving inference efficiency. We conduct experiments on three datasets. In most cases, the forecast results were significantly better than other baselines, and the accuracy of causal discovery was not significantly different from that of SOTA.