Resilience driven EV coordination in multiple microgrids using distributed deep reinforcement learning

By integrating electric vehicles (EVs), the multi-microgrids (MMGs) can significantly enhance their resilient operation capabilities. However, existing works face challenges in formulating optimal routing and scheduling strategies for EVs, due to the spatial-temporal uncertainty of the distribution and transportation networks, as well as incomplete information. This paper addresses the coordination problem of EVs for the resilience enhancement of MMGs, using a distributed multi-agent deep reinforcement learning approach to minimize the load-shedding cost. Specifically, a coupled power-transportation network (CPTN) model is constructed to facilitate EV routing and scheduling for resilience enhancement, considering the uncertainties associated with distributed renewables, load profiles, and traffic flow. Then, the coordination problem of each EV is formulated as a partially observable Markov decision process, and an attention-based distributed multi-agent deep deterministic policy gradient method, namely AD-MADDPG, is proposed to learn the optimal strategies. The proposed method applies an architecture with multi-actor, single-learner to reduce training complexity, employing a convolutional neural network to capture spatial characteristics from the CPTN, and incorporating a long short-term memory to derive temporal sequence features across multiple time steps, thereby enhancing the exploration efficiency of the action space. Simulation results implemented on the modified IEEE 33-bus test feeder demonstrate that AD-MADDPG outperforms all other baselines in terms of load restoration, restoration fairness, and energy consumption when varying different numbers of EVs, maximum discharging proportion, and maximum moving distance.