PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
February 2, 2026International Journal of Neural Systems0 citations

Causal Mask in Transformer via Transfer Entropy from Vector Autoregressive Learning for Multivariate Time Series Forecasting

View Full Paper
CZChengli ZhouYHYaqun HuangDTDapeng Tao

Key Points

  • The aim is to enhance time series forecasting by integrating causal discovery with a transformer model.
  • Proposed ARCausal framework combines transfer entropy and transformer attention models.
  • Introduced a causal masking mechanism derived from transfer entropy.
  • Used vector autoregression to refine causal estimations.
  • Conducted experiments on nine benchmark datasets.
  • Achieved a 4% reduction in overall mean squared error (MSE) compared to baseline methods.
  • Demonstrated high predictive accuracy and interpretability through visualization analyses.

Abstract

Time series forecasting poses a significant challenge across diverse domains such as finance and climate science, yet existing deep learning models often emphasize sequential dependencies while neglecting causal structures, leading to spurious correlations and reduced interpretability. To address these limitations, a novel time series forecasting framework, ARCausal, that explicitly integrates Transfer entropy–based causal discovery with Transformer attention modelin is proposed. Unlike prior causal inference approaches that rely on static assumptions or expensive graph construction, ARCausal introduces a sparse a sparse causal masking mechanismderived from transfer entropy and refined through vector autoregression (VAR)-guided estimation to efficiently capture dynamic causal interactions. This causal mask suppresses non-informative dependencies and enforces clear separation between autocorrelation and true causal effects, enabling both high predictive accuracy and interpretability. Extensive experiments on nine benchmark datasets (including ETT, Weather, and Exchange) demonstrate that ARCausal achieves pleasant performance, reducing the overall MSE by 4% compared to baseline methods while maintaining computational efficiency. Visualization analyses further confirm that our causal masks yield interpretable structures. The implementation is publicly available at https://github.com/jancely/ARCausal .

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

Zhou et al. (2026) studied this question.

synapsesocial.com/papers/6980fe9bc1c9540dea810cbahttps://doi.org/10.1142/s012906572650019x
Ask AI
Helpful
Bookmark
Share
View Full Paper