Distributed Training and Distributed Execution-Based Stackelberg Multi-Agent Reinforcement Learning for EV Charging Scheduling

Multi-agent deep reinforcement learning (MADRL) has been applied to EV charging scheduling. However, it relies on centralized training and thus is significantly challenged by privacy, communication and computing concerns. To address this issue, this letter proposes a novel distributed training and distributed execution (DTDE) framework-based fully-distributed Stackelberg multi-agent deep reinforcement learning (SMADRL) approach for EV charging scheduling. It models the agents’ asymmetry competition as a novel Stackelberg Distributed Partially-Observable Markov Decision Process (SDis-POMDP). This addresses the privacy and communication concerns as the agents do not need to share local observations, enhances the computational efficiency and scalability as all calculations are conducted at local agents, and does not rely on high-precision system modeling which is required by model-based methods. The effectiveness and efficiency of the proposed strategy are verified by comparing its performance against multiple benchmarks.