Soft actor-critic energy management in three-phase unbalanced microgrids with lagrangian penalty constraints

Efficient energy management in microgrids remains challenging due to the stochastic variability of renewable generation and the inherent electrical characteristics of distribution networks. Unbalanced three-phase AC microgrids exhibit complex operating conditions driven by unequal loading, asymmetrical line parameters, and electromagnetic coupling effects, which complicate control design under uncertainty. This work presents a reinforcement learning–based energy management system built upon the Soft Actor–Critic (SAC) algorithm, specifically tailored for unbalanced three-phase microgrids. To ensure compliance with operational limits, a Lagrangian penalty mechanism is incorporated to regulate voltage magnitudes and other technical constraints during training. The proposed approach is benchmarked against a deterministic quadratic constrained programming (QCP)–based energy management strategy under stochastic operating scenarios. Extensive Monte Carlo simulations demonstrate that the SAC-based controller achieves higher typical operational returns while maintaining voltage levels within admissible limits and only marginally increasing phase unbalance compared to the optimization-based baseline. Although the QCP strategy exhibits lower per-step computational times, both approaches remain computationally feasible for practical deployment. These results suggest that reinforcement learning can serve as a flexible alternative for energy management in realistic unbalanced three-phase microgrids operating under uncertainty.