Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting

Extending the forecasting time is a critical demand for real applications, as extreme weather early warning and long-term energy consumption. This paper studies the long-term forecasting problem of time series. Transformer-based models adopt various self-attention mechanisms to the long-range dependencies. However, intricate temporal patterns of long-term future prohibit the model from finding reliable dependencies., Transformers have to adopt the sparse versions of point-wise-attentions for long series efficiency, resulting in the information bottleneck. Going beyond Transformers, we design Autoformer as a decomposition architecture with an Auto-Correlation mechanism. We break the pre-processing convention of series decomposition and renovate it as a inner block of deep models. This design empowers Autoformer with decomposition capacities for complex time series. Further, inspired the stochastic process theory, we design the Auto-Correlation mechanism on the series periodicity, which conducts the dependencies discovery and aggregation at the sub-series level. Auto-Correlation self-attention in both efficiency and accuracy. In long-term, Autoformer yields state-of-the-art accuracy, with a 38% relative on six benchmarks, covering five practical applications: energy,, economics, weather and disease. Code is available at this repository:\{https://github.com/thuml/Autoformer}.