BlindLight: High Robustness Reinforcement Learning Method to Solve Partially Blinded Traffic Signal Control Problem

Adaptive traffic signal control plays a crucial role in enhancing the traffic situation in urban cities. Recently, reinforcement learning-based methods have demonstrated remarkable performance in addressing traffic signal control problems. However, the deployment of these methods in real-world environments is limited due to their lack of robustness. When errors occur during traffic data collection, their performance tends to deteriorate significantly. This paper addresses a prevalent issue where certain intersections, referred to as blinded intersections, cannot receive traffic data either due to hardware failures or the absence of traffic detectors. Existing adaptive traffic signal control methods fail to achieve satisfactory results in such scenarios as they are unable to learn effective policies for blinded intersections. Through theoretical analysis, we identify the primary reason behind the poor performance of existing methods as improper reward selection. To mitigate this issue, we propose a novel reward function called NOVO, which incorporates Number of Vehicles and Outflow, offering a correct optimization objective with low variance. These advantages significantly enhance the performance of reinforcement learning methods. Additionally, we introduce a new reinforcement learning model called BlindLight, which employs a dual model structure to learn Q-values for different types of intersections independently. This model design enhances the robustness of the system. Experimental results conducted on public datasets demonstrate the consistent performance improvement achieved by the theoretically supported reward function NOVO in existing methods with blinded intersections. Furthermore, BlindLight outperforms all state-of-the-art traffic signal control methods significantly.