Event-Based Deep Reinforcement Learning for Smoothing Ramp Events in Combined Wind-Storage Energy Systems

Large-scale wind power ramp events will significantly affect the power balance and frequency stability of the power grid, endangering the economy and stability of the power grid. In this article, an event-based soft actor-critic (EB-SAC) algorithm using an event-driven mechanism is proposed to optimize the ramp event smooth scheduling of the wind-storage combined system composed of the wind farm and pumped storage power station. First, the wind power energy ramp event model and the pumped energy model are established. The kernel density estimation method is used to simulate the probability distribution of wind power prediction error, and the training data set considering wind power prediction error is obtained by probability sampling. Then, the author defines a comprehensive value function considering multiple factors such as wind power ramp event smoothing demand, pumped storage operation characteristics, energy storage capacity status, and hourly electricity price. Second, the event-driven mechanism is added to the traditional SAC optimization process to realize the intermittent decision-making scheduling optimization of energy storage based on wind power energy events, which effectively reduces the number of power changes of pumped storage power stations. Finally, the simulation is verified according to the actual wind farm operation data. By comparing with the smoothing results of the other algorithm, the results show that the EB-SAC algorithm has better optimization performance for the smoothing optimization of wind power ramp events, effectively reduces the number of pumped storage action changes, and improves the efficiency and benefit of ramp event smoothing.