Model-Free Emergency Frequency Control Based on Reinforcement Learning

Unexpected large power surges will cause instantaneous grid shock and, thus, emergency control plans must be implemented to prevent the system from collapsing. In this article, with the aid of reinforcement learning, novel model-free control (MFC)-based emergency control schemes are presented. First, multi-Q-learning-based emergency plans are designed for limited emergency scenarios by using offline-training-online-approximation methods. To solve the more general multiscenario emergency control problem, a deep deterministic policy gradient (DDPG) algorithm is adopted to learn near-optimal solutions. With the aid of deep Q network, DDPG-based strategies have better generalization abilities for unknown and untrained emergency scenarios, and thus are suitable for multiscenario learning. Through simulations using benchmark systems, the proposed schemes are proven to achieve satisfactory performances.