Spatial–Temporal Interaction–Aware Multiagent Reinforcement Learning for Traffic Signal Control

Multi-intersection traffic signal control (MTSC) plays a crucial role in intelligent transportation systems. Multiagent reinforcement learning (MARL) has shown great promise in enabling MTSC. However, existing MARL-based MTSC approaches primarily focus on capturing the spatial relationships among intersections, often neglecting the dynamic spatiotemporal correlations that emerge from fluctuating traffic flows. In MTSC scenarios, traffic flow propagates across intersections over time, indicating that the historical traffic information of intersections inherently reflects the spatial relationships among traffic signals. To address this, we propose a novel algorithm called spatiotemporal interaction–aware multiagent reinforcement learning (STIA-MARL), which leverages historical traffic flows to effectively learn the dynamic relationships among intersections. From a modeling perspective, we construct a new graph structure tailored to the traffic network, altering the adjacency relationships at the model level, to improve the adaptability of the shared policy network to different intersections and enhance model performance. Experimental results on both synthetic and real-world data sets demonstrated that STIA-MARL achieved at least an 11.5% improvement in average travel time compared with state-of-the-art methods.