An adaptive quasi-PR controller for modular multilevel converters based on deep reinforcement learning

In modern power systems, Modular Multilevel Converter (MMC) plays an important role due to its advantages of convenient maintenance and easy expansion. However, because of its structural defects, it is easy to induce circulating current, which poses a challenge to stable operation. It is of great significance to develop efficient and reliable circulating current suppression technology. This paper introduces a Deep Reinforcement Learning (DRL) method for adaptive tuning of controller parameters, addressing the challenge of difficult parameter adjustment in the MMC circulating current suppression strategy employing a quasi-PR controller. It analyzes the feasibility of using the twin delayed deep deterministic (TD3) algorithm to tune the parameters of the PR controller, and designs a reasonable neural network and reward function to train the agent for control. Simulation results demonstrate the superiority of the TD3-based adaptive quasi-PR controller over the traditional fixed-parameter quasi-PR controller. The adaptive controller has a better effect on MMC circulating current suppression, with better dynamic response and smaller THD. This provides an effective solution for promoting the large-scale application of MMCs and enhancing the performance of power systems.