Intra-day dispatch method via deep reinforcement learning based on pre-training and expert knowledge

This is the authors' abstract. We don't add key points for this paper.

• An initial policy network training method based on pre-training with supervised learning is proposed. • The proposed significantly speeds up the training process of deep reinforcement learning and greatly reduces the model development cycle. • Expert knowledge is embedded in the deep reinforcement learning to guide the training of the agent. • The agent quickly learns to limit the search direction to the feasible region of the power system operation to improve the convergence. • To obtain high rewards, agent learns to prioritize the renewable energy utilization which significantly reduces the curtailment rate. Traditional economic dispatch algorithms rely on the accuracy of all parameters and also lack the adaptability to the high uncertainties brought by the dynamic changes happening in the current power systems. Its computing efficiency also needs to be improved with the increased operational complexities. In recent years, due to high self-learning and self-optimization ability, reinforcement learning has emerged in the field of economic dispatch, which can solve model-free dynamic programming problems that cannot be effectively solved by traditional optimization methods. In this paper, we construct a reinforcement agent for intra-day dispatch to optimize generator output, using a twin delayed deep deterministic policy gradient algorithm based on pre-training and expert knowledge (PEK-TD3). Aiming at solving the problems of long exploration time and poor convergence of conventional deep reinforcement learning, we propose an initial policy network training method based on pre-training with supervised learning, which significantly speeds up the training process of deep reinforcement learning and greatly reduces the model development cycle. At the same time, expert knowledge is embedded in the deep reinforcement learning to guide the training of the agent. With the guidance of expert knowledge, on the one hand, the agent quickly learns to limit the search direction to the feasible region of the power system operation so as to improve the convergence. On the other hand, in order to obtain higher rewards, agent learns to prioritize the renewable energy utilization which significantly reduces the curtailment rate of renewable energy. Finally, the modify IEEE 118-node system is used to verify the performance of the proposed method.