Vehicle trajectory planning under observational perturbations with robust adversarial reinforcement learning

Vehicle trajectory planning is a critical component in the signal-vehicle cooperative control (SVCC) system. Reinforcement learning (RL)-based methods show great advantages in addressing the complexity of the trajectory planning problems, particularly in adapting to dynamic traffic environments, interacting with surrounding road users, and optimizing long-term performance. Nevertheless, ensuring the robustness of RL generated control policies remains a key challenge, especially under observational perturbations such as perception errors, and malicious cyber attacks. This paper introduces a robust and safe RL-based vehicle trajectory planning model under observational perturbations. The trajectory planning problem is first formulated as a Constrained Markov Decision Process (CMDP), to incorporate safety metrics as costs in the RL framework. Then, a systematic approach is developed to construct both effective and stealthy perturbations. We first prove the existence of an optimal adversary under bounded observational perturbations for each objective defined on the value functions, and then identify the perturbation direction using the gradients of value functions, and determine the perturbation magnitude using Shapley analysis. To generate control policies that achieve robust performance under observational perturbations, a safety gated adversarial RL framework is developed. Experimental results from both synthetic and real world arterial corridors demonstrate that the proposed adversaries substantially degrade system performance in both safety and fuel consumption. Furthermore, adversarial training significantly enhances the robustness of RL agents against various types of observational perturbations.