Adaptive Traffic Control Based on P-DQN+HER+TD3: Toward Real-World Deployments

Urban traffic congestion adversely impacts vehicle mobility, fuel consumption, and environmental sustainability. Classical traffic-control schemes—fixed-time and actuated methods—suffer from rigidity and limited responsiveness, failing to adapt to dynamic, stochastic demand or capture network-wide spatiotemporal correlations. Although reinforcement learning has emerged as a promising alternative, prevalent discrete-only approaches constrain controllers to coarse-grained phase selection, yielding suboptimal performance under continuous traffic dynamics. To address these limitations, this study proposes a hybrid deep reinforcement learning framework integrating a Parameterized Deep Q-Network, Twin Delayed Deep Deterministic Policy Gradient, and Hindsight Experience Replay for intelligent traffic-signal optimization. The model jointly handles discrete-phase selection and continuous-phase duration within a unified parameterized action space. An analytical state-dependent queue model grounds the framework in traffic-flow theory, capturing dynamic queue transitions and server capacities. Both synthetic traffic data and preprocessed Next Generation Simulation trajectory data derive state variables including density, speed, queue length, and delay. The framework is evaluated in an intersection simulation against PressLight, Max-Pressure, and Frame-based Reinforcement Learning with All-Phase competition models. The proposed approach achieves lower delay, queue length, and travel time with higher throughput. These findings demonstrate that integrating trajectory-based state modeling, analytical queue formulations, and hybrid discrete–continuous reinforcement learning offers an effective and scalable solution for real-world adaptive traffic-signal control.