Hierarchical deep reinforcement learning for resilient de-icing in power-traffic coupled distribution networks

Frequent extreme ice storms threaten the operational resilience of modern distribution networks. While coordinating Soft Open Points (SOPs) and Mobile De-icing Devices (MDIDs) offers a flexible spatiotemporal de-icing solution, jointly optimizing MDID traffic routing (discrete) and SOP power control (continuous) constitutes a complex bi-level problem. Traditional heuristics suffer from unacceptable online computational delays, and standard single-agent reinforcement learning struggles with gradient conflicts in mixed-action spaces. To address this, we propose a novel Goal-Conditioned Hierarchical Reinforcement Learning (HRL) framework. A Manager agent optimizes macroscopic MDID routing, while a Worker agent simultaneously regulates high-frequency SOP power flows to prevent cascading failures and electrical over-limits. Simulations on a 114-bus power-traffic coupled network demonstrate that HRL guarantees stable convergence, achieves millisecond-level real-time execution, and exhibits exceptional dynamic robustness against sudden traffic disturbances, significantly enhancing smart grid resilience.