Enhancing Robustness of Deep Reinforcement Learning Based Adaptive Traffic Signal Controllers in Mixed Traffic Environments Through Data Fusion and Multi-Discrete Actions

With the rapid development of artificial intelligence (AI) and connected vehicle (CV) technology, researchers are actively exploring the utilization of deep reinforcement learning (DRL) algorithms combined with real-time traffic information from CVs to optimize traffic signal control. These controllers have showcased better performance than traditional controllers. However, a major drawback is their heavy reliance on pure CV environments, which has not been adequately addressed. This study proposes a novel traffic signal controller based on proximal policy optimization (PPO), integrating a multi-discrete action space and a combined state space, to enhance robustness in mixed traffic environments where CVs and non-connected vehicles coexist. Evaluations through simulation experiments on a real-world-based intersection testbed demonstrate superior performance in terms of both effectiveness and robustness compared to some popular controllers, including the deep Q-network (DQN) based controller, pretimed controller, and actuated controller. The results indicate that the proposed controller significantly reduces the average delay. Furthermore, its performance remains reliable even in environments with a CV market penetration rate as low as 20%. The findings highlight that the utilization of PPO with multi-discrete actions and combined state space effectively addresses the challenges posed by mixed traffic environments, making it a promising solution for real-world traffic signal control.