Deep Reinforcement Learning-Based Dynamic Droop Control Strategy for Real-Time Optimal Operation and Frequency Regulation

The optimal operation of an islanded AC microgrid system is achieved by proper power sharing among generators. The conventional distributed cost optimization strategies use a communication system to converge incremental costs. However, these methods are dependent on the distributed communication network and do not consider frequency deviations for real-time load variability. Thus, this paper proposes a DRL-based dynamic droop control strategy. The proposed twin delayed DDPG-based DRL interacts with the environment to learn the optimal droop gain for reducing generation cost and frequency deviation. The trained agent uses local information to transmit dynamic droop gains to the primary controller as demand load changes. It can simplify the control structure by omitting the secondary layer for optimal operation and power quality. The proposed control strategy is designed with a centralized DRL training process and distributed execution, enabling real-time distributed optimal operation. The comparison results with conventional distributed strategy confirms better control performance of the proposed strategy. Finally, the feasibility of the proposed strategy was verified by experiment on AC microgrid testbed.