Security Risks of AI/ML With a Focus on Reinforcement Learning: A Review and Perspectives From Grid Applications

The increasing reliance on artificial intelligence (AI) and machine learning (ML)-based methods across various domains including safety-critical applications, e.g. energy systems, self-driving cars, has introduced significant security risks due the vulnerabilities of AI/ML methods to adversarial attacks. Among various forms of adversarial attacks, poisoning attacks, which manipulate the training environment, pose a severe threat to the integrity and security of the trained AI/ML models. This paper first highlights the security threats of general machine learning (ML)-based methods considering different stages of the end-to-end ML pipeline that are vulnerable to adversarial attacks. Next, we provide comprehensive review on poisoning attacks with a specific focus on their effects in reinforcement learning (RL)-based methods. We explore observation, policy, reward poisoning and backdoor attacks on RL, and discuss the induced implications on smart energy grid applications, specifically considering market, systems, controls, and communication perspectives. Furthermore, we examine defense mechanisms and mitigation strategies that have been proposed to safeguard RL-driven energy infrastructures from these adversarial threats. By analyzing these attacks from generic AI/ML perspective with a focus on RL as well as grid application viewpoint, this paper provides insights into current challenges and future research directions advancing the robustness of RL-based methods against such poisoning attacks, particularly in their applications in power and energy systems.