Autonomous Voltage Regulation for Smart Distribution Network With High-Proportion PVs:A Graph Meta-Reinforcement Learning Approach

Smart distribution grids with a high percentage of distributed photovoltaic (PV) systems often face severe voltage quality problems. Deep Reinforcement Learning (DRL) presents great potentials in achieving optimal voltage level control in distribution grids with the advantage of learning without explicit modeling. However, when DRL is applied to active voltage regulation in distribution grids with a high proportion of distributed photovoltaic (PV) power generation, it encounters several challenges, such as uneven learning among controllable devices and environmental instability, which impede the convergence of DRL. In addition, traditional reinforcement learning represents the distribution network with vectors and ignores topology information. In this paper, we transform the voltage control problem into a partially observable Markov decision process, and propose a multi-agent meta-reinforcement learning algorithm based on graph convolutional networks to improve voltage quality by combining DRL with meta-learning (ML). By integrating ML, our approach allows agents to better predict the actions of other agents, thus mitigating the instability of the environment. To address the problem of uneven learning among agents, we incorporate a self-concern mechanism and a value decomposition method into the critique network. The proposed algorithm is validated through experiments on IEEE 33-bus, 141-bus, and 322-bus systems and compared with five other multi-agent DRL methods and model-based model predictive control algorithms.