A network-aware hybrid deep reinforcement learning optimization framework for microgrid energy management under uncertainty

This is the authors' abstract. We don't add key points for this paper.

Coordinating operating cost, network losses, and operational security under uncertain load and photovoltaic (PV) generation is a key challenge in distribution-system energy management. This paper presents a two-stage hybrid day-ahead energy management system that combines a deep Q-network (DQN) with second-order cone quadratic programming (SOCQP) for modified IEEE 13-bus and 33-bus systems. The DQN was trained on 92 daily trajectories selected from January, July, and November 2017 and evaluated on three unseen days in March, August, and December. Training across four random seeds showed consistent convergence after approximately 2000 episodes. In Phase I, the trained agent determines generator commitment and coarse dispatch decisions. In Phase II, the SOCQP refines these decisions while enforcing nodal power balance, voltage, branch-flow, generator, and line-loss constraints. The proposed method is benchmarked against a deterministic perfect-information MISOCQP using realized load and PV data and a stochastic MISOCQP based on 1000 Monte Carlo scenarios reduced to 10 representative scenarios. Across the three test days, the SOCQP refinement reduced the raw DQN peak generation by between 0.15% and 5.81% for the 13-bus system and between 5.66% and 9.90% for the 33-bus system. Over the interior cost and loss weight settings, the 13-bus hybrid cost was, on average, approximately 2.7% above the deterministic and stochastic results, while its line loss was about 23.5% lower than that of the stochastic formulation. For the 33-bus system, the mean absolute cost difference between the hybrid and stochastic approaches was approximately 2.4%, while the hybrid loss profile remained within about 6.1% of the perfect-information reference, although the stochastic formulation obtained the lowest absolute losses. Since the deterministic model uses realized trajectories, whereas the uncertainty-aware methods use synthesized predictions, it is treated as an ideal reference rather than a directly comparable operational case. Following offline training, for each test day, the complete hybrid workflow solved the 24-hour scheduling problems in under 2 s and 5 s for the 13-bus and 33-bus systems, respectively, compared with approximately 34 and 113 min for the stochastic benchmark. The results show that the proposed framework combines uncertainty-responsive scheduling, network-feasible dispatch refinement, competitive cost and loss performance, and substantially lower online computational requirements.