Multi-Agent Deep Reinforcement Learning for V2G Scheduling and Incentive Mechanism to Enhance Distribution Network Loadability

The rapid integration of distributed renewable energy has substantially increased the operational uncertainty of distribution networks, making the preservation of sufficient loadability increasingly difficult. To address this problem, a bi-level coordinated scheduling framework is developed in which the interaction between the distribution network operator (DNO) and electric vehicle aggregators (EVAs) is modeled as a Stackelberg game. Unlike existing studies that treat supply security only as a constraint, allocate revenue solely by economic contribution, and prioritize experience replay by TD error alone, the proposed framework embeds the loadability directly into the upper-level objective, allocates the aggregated revenue by an improved Shapley value method that combines an S-shaped utility function reflecting user participation with a loadability contribution factor quantifying each user’s contribution to supply security, and solves the bi-level model by a dual-agent Soft Actor–Critic (SAC) and Multi-Agent Twin Delayed Deep Deterministic Policy Gradient (MATD3) framework equipped with a Stackelberg-Aware Dynamic Prioritized Experience Replay (SD-PER) mechanism that augments the TD error with opponent strategy variation to make the replay aware of the leader–follower hierarchy. Case studies on the IEEE 33-bus system and a real 72-bus distribution network in Shanghai show that the proposed method raises the average peak-period loadability from 1.05 to 1.24 on the 33-bus system, lifts the minimum nodal voltage at the evening peak from 0.937 to 0.953 p.u. while reducing the average voltage-deviation index by about 20.5%, lowers the net user cost on the EVA side, and attains a scheduling quality within 1% of a KKT/MPEC bi-level benchmark, while the SD-PER mechanism accelerates convergence to about 1440 episodes and improves the final reward by about 40% over the base SAC. The same advantages are reproduced on the real distribution network in Shanghai, confirming the practical applicability and scalability of the method.