IBformer: Inductive Bias is Necessary for Multivariate Time Series Forecasting

Transformers use a powerful self-attention mechanism to model remote dependency capabilities, showing great potential in various long-term time series prediction tasks. However, the original Transformer was not proposed for the time series prediction tasks and did not make any assumptions about temporal patterns and characteristics. Due to systematic differences and external factors, the heterogeneity of multi-dimensional time series data sets makes Transformer prediction face challenges. A large number of researchers lack inductive bias when modeling multi-dimensional time series and dealing with different structures or patterns. In this work, we addressed the heterogeneity of multivariate time series from both temporal and spatial perspectives, examined the characteristics of multivariate time series and the inherent biases of different models, and proposed IBformer, a time series Transformer with inductive bias. Specifically, we design a key sampling strategy to extend the receptive field of the model by introducing inductive bias, and perform discriminative modeling for different datasets based on decomposition, utilizing encoders and trend prediction blocks to capture seasonal patterns and overall trends, solving the problem of temporal heterogeneity. In addition, we adopt a channel-independence strategy and design a quadratic spatio-temporal enhancement block to correct the original prediction results, which solves the problem of weak temporal correlation caused by direct multi-step prediction. And capture spatial dependence on the premise of avoiding excessive spatial modeling, and solve the problem of spatial heterogeneity. Extensive experiments on seven real-world datasets show that IBformer outperforms all existing models and demonstrates strong versatility and generalization capabilities. The code is made available at https://github.com/shihaoyuan6/IBformer