Transformer Network-Based Probabilistic Distribution Prediction for Electricity Load

Abstract Traditional electricity load forecasting methods struggle to address renewable energy fluctuations and long-term dependencies. This paper proposes a Transformer-quantile regression hybrid model to improve prediction accuracy and confidence interval reliability. To overcome the inefficiency of Recurrent Neural Networks (RNNs) in processing long sequences and their poor cross-cycle feature extraction, we implement multi-head self-attention mechanisms for parallel global temporal modeling, coupled with a differentiable sorting loss to enforce quantile monotonicity. Experimental results show a Mean Absolute Percentage Error (MAPE) of 0.0318 (10.2% lower than Long Short-Term Memory networks (LSTM)) using 56-day data, with 89.3% prediction interval coverage and 22% width reduction. By leveraging Transformer’s parallel computation and dynamic encoding, this framework delivers efficient and stable forecasts for highly volatile power systems.