Constraint-Guided Reinforcement Learning for Microgrid Energy Storage Scheduling Motivated by Virtual Power Plant Applications

The increasing integration of renewable energy and energy storage systems presents challenges for microgrid scheduling under uncertain operating conditions and multiple operational constraints. Conventional reinforcement learning methods may generate infeasible actions, while direct safety clipping can create inconsistencies between policy outputs and executed actions, affecting learning efficiency and scheduling performance. To address these issues, this paper proposes a constraint-guided safety-correction soft actor–critic (CGSC-SAC) algorithm for microgrid energy storage scheduling with a view to virtual power plant applications. The proposed method integrates executed-action replay, critic action clipping, and a correction-guided actor loss to incorporate operational constraints into policy learning and reduce the discrepancy between generated and feasible actions. Comparative experiments, ablation studies, and robustness tests are conducted in a grid-connected photovoltaic–battery system. The results show that CGSC-SAC improves the mean reward from −116.358 to −114.091 and reduces cumulative safety correction by 67.9% compared with Safe-Clip-SAC. Under supply shortages and renewable-energy surplus conditions, safety correction is reduced by 64.7–68.4% and 59.3–64.5%, respectively. Furthermore, an extended IEEE 33-bus case is investigated to evaluate the applicability of the proposed method in a multi-node microgrid. The results demonstrate comparable load satisfaction and no battery state-of-charge violations across the tested scenarios, with computational efficiency suitable for hourly scheduling. These findings indicate that CGSC-SAC improves constraint-aware energy storage scheduling and provides a potential solution for local resource management in virtual power plant applications.