Optimizing multi-timescale market participation for distributed renewable energy: A Bi-level model predictive control and improved reinforcement learning framework

With the rapid development of distributed renewable energy, supply–demand imbalances and stochastic fluctuations across multiple timescales in power systems have become increasingly prominent. This paper proposes a framework for market participation by distributed renewable energy aggregators (DREAs) based on nested bi-level model predictive control (MPC) and improved reinforcement learning. In the upper layer, weather-state and unit operating-state labels are incorporated into a Wasserstein generative adversarial network with gradient penalty to generate joint wind–PV output data. These data are then used to support rolling coordinated optimization across the medium- and long-term, day-ahead, and tradable green certificate markets, while real-time deviation information fed back from the lower layer is incorporated to dynamically adjust subsequent trading plans. In the lower layer, real-time deviations are corrected through real-time market participation, self-consumption load adjustment, and energy storage management, while real-time operating states and deviations are fed back to the upper layer, forming a prediction–optimization–execution–feedback loop. The artificial intelligence contribution is the proposed Soft Actor-Critic with Adversarial Reinforcement Learning (SAC-ARL) algorithm, which introduces state-perturbation training and policy-smoothing regularization to improve policy stability under uncertain conditions such as forecast deviations and cyberattacks. From an engineering perspective, system-wide coordinated trading strategies are developed for DREAs participating across multiple market types and timescales. Simulation results show that, compared with the benchmark scheme, the proposed scheme reduces real-time deviations by 37.3% and real-time market regulation costs by 69.75%, while increasing net profit by 4.37%, verifying its effectiveness and engineering applicability.