GeoSAT: long-term time series forecasting with GeoSparse attention and tempo-spatial feature extraction

Long-term time series forecasting (LTSF) holds a foundational role in numerous real-world scenarios such as energy monitoring and industrial equipment scheduling, where forecasting systems frequently operate under limited computational resources and require real-time responsiveness. However, existing methods often struggle to precisely extract fine-grained temporal patterns, including long-range trends and cross-variable dependencies, while maintaining a balance between prediction accuracy and computational efficiency in practical deployments. In tackling this challenge, this paper proposes a lightweight LTSF model named GeoSAT that utilizes geometric sparse (GeoSparse) attention and a tempo-spatial feature extraction module. The model first decouples the input time-series data into coarse-grained trend and residual components. For the residual component, a stacked stationary wavelet transform (SSTW) is employed to transform the embedded time domain into a multi-scale frequency domain that are then stacked into a unified frequency-domain representation, enabling the model to capture fine-grained temporal patterns at different scales. Then, compared to traditional sparse attention that relies solely on dot-product, we introduce the GeoSparse attention mechanism applied to the unified frequency-domain representation. It preserves the dot product and introduces the wedge product to effectively capture scalar similarities across different scales as well as geometric correlations between variables. Through sparsification, it highlights critical frequency-domain information and reduces time complexity. For the trend component, a tempo-spatial feature extraction module is employed to model the spatiotemporal domain through adaptive temporal-frequency filtering within variables and one-dimensional convolution between variables. Residual connections are used to extract both the temporal and spatial patterns, thereby capturing the global trend and cross-variable relationships. Evaluations on eight public datasets demonstrate that the proposed model effectively capture the fine-grained features and outperform traditional LTSF methods while ensuring a lightweight architecture. The code is available at: https://github.com/IntelligentComp-Lab/GeoSAT.