Single-Attention Large Language Model for Efficient Multi-Regional Electricity Demand and Generation Forecasting

Electricity forecasting is one of the most crucial aspects in maintaining stable, reliable, and autonomous power systems. While recently developed forecasting methods based on large language models can make accurate predictions, these models are still struggling due to their computational complexity, which requires more computing power, and their reliance on carefully designed prompts. This makes them complicated and harder to use in practice. To address this, we propose a Single-Attention Large Language Model (SA-LLM) that uses a unified attention mechanism to understand relationships between main and additional variables, without the need for manually created prompts. The proposed framework has been tested on real electricity supply and demand datasets, which are obtained from major U.S. electricity markets, including PJM, MISO, NYISO, ISO New England, ERCOT, SPP, and CAISO. Experimental results demonstrate that the proposed SA-LLM method outperforms the existing counterpart methods in terms of accuracy and associated errors. More specifically, the SA-LLM has also achieved a 22.5% improvement in the mean absolute error compared with traditional LSTM-based models, while reducing memory usage by 52.1% and training time by 38.4% relative to recent LLM-based methods. Furthermore, the SA-LLM demonstrates strong zero-shot generalization, achieving an additional 18.2% improvement in the MAE on previously unseen regions.