Improvise, Adapt, Overcome: Dynamic Resiliency Against Unknown Attack Vectors in Microgrid Cybersecurity Games

Cyber-physical microgrids are vulnerable to rootkit attacks that manipulate system dynamics to create instabilities in the network. Rootkits tend to hide their access level within microgrid system components to launch sudden attacks that prey on the slow response time of defenders to manipulate system trajectory. This problem can be formulated as a multi-stage, non-cooperative, zero-sum, Markov game with the attacker and the defender modeled as opposing players. To maximize defender utilities at all stages of the game, this paper proposes a deep reinforcement learning-based strategy that dynamically identifies rootkit access levels and isolates incoming manipulations by incorporating changes in the defense plan. Our proposed approach introduces a novel reward formulation mechanism that quantifies the level of microgrid instability in real time. This allows us to detect the presence of rootkits in the microgrid. The developed reward formulation also allows us to achieve scalability of the proposed solution to different grid sizes and configurations. A major advantage of the proposed strategy is its ability to establish resiliency without altering the physical transmission/distribution network topology, thereby diminishing potential instability issues. The paper presents several simulations and case studies to demonstrate the operating mechanism and robustness of the proposed strategy.