Reinforcement Learning Solution for Cyber-Physical Systems Security Against Replay Attacks

The security problem of state estimation plays a critical role in monitoring and managing operation of cyber-physical systems (CPS). This paper considers the problem of network security under replay attacks and formulates a novel attack detection method. More specifically, we design a model-free reinforcement learning-based replay attack detection framework that can automatically learn and recognize the evolving attacks with more effectiveness. Attackers in some situations are more like intelligent agents with initiative, who can transform their attack strategies purposefully according to the actions of defenders. Thus, we propose a new defense strategy against the interaction between the attacker and the defender which is solved by optimization learning. The proposed analytical procedure concerning reinforcement learning technology can also be extended to the study of other control applications. Finally, the numerical examples are provided to illustrate the effectiveness of the detection method.