Adversarially Robust Reinforcement Learning for Energy Management in Microgrids with Voltage Regulation Under Partial Observability

Modern microgrids increasingly rely on learning-based energy management systems (EMSs) for real-time decision-making, yet remain vulnerable to cyber–physical disturbances, sensor tampering, and model uncertainty. Existing resilient control and robust reinforcement learning methods provide useful foundations, but rarely address adversarial measurement perturbations that distort belief evolution under partial observability. This gap is critical, as structured perturbations in sensing channels can destabilize learning-based policies and propagate into voltage-regulation violations. This paper proposes an adversarially robust reinforcement learning framework for energy management with voltage regulation under partial observability in microgrids. The EMS decision-making problem is formulated as a partially observable Markov decision process (POMDP) that accounts for adversarial measurement perturbations, belief evolution, and system-level economic and voltage constraints. To avoid excessive conservatism under worst-case uncertainty, an adversary-aware belief construction based on adversarial belief balancing (A3B) is employed to focus on policy-relevant perturbations. Building on this belief representation, an adversarially robust learning framework is developed by incorporating adversarial counterfactual error (ACoE) as a learning regularization mechanism, enabling a balance between nominal operating efficiency and robustness under adversarial measurement distortion. The case study is conducted on a medium-voltage radial distribution feeder (IEEE 123-Node Test Feeder). Case study results demonstrate that the proposed ACoE-regularized policies substantially reduce voltage-deficit events, improve policy stability, and maintain operational constraints under adversarial perturbations, consistently outperforming standard proximal policy optimization (PPO)-based controllers. These results indicate that counterfactual-aware, belief-based learning substantially enhances voltage quality and operational resilience in microgrids with high penetration of distributed energy resources.