Deep reinforecement learning based optimal defense for cyber-physical system in presence of unknown cyber-attack

In this paper, the online optimal cyber-defense problem has been investigated for Cyber-Physical Systems (CPS) with unknown cyber-attacks. Firstly, a novel cyber state dynamics has been generated that can evaluate the real-time impacts from current cyber-attack and defense strategies effectively and dynamically. Next, adopting game theory technique, the idea optimal defense design can be obtained by using the full knowledge of cyber-state dynamics. To relax the requirement about cyber-state dynamics, a game-theoretical actor-critic neural network (NN) structure was developed to efficiently learn the optimal cyber defense strategy online. Moreover, to further improve the practicality of developed scheme, a novel deep reinforcement learning algorithm have been designed and implemented into actor-critic NN structure. Eventually, the numerical simulation demonstrate that proposed deep reinforcement learning based optimal defense strategy cannot only online defend the CPS even in presence of unknown cyber-attacks, and also learn the optimal defense policy more accurate and timely.