Pressure-Guided Q-Learning: a warm-started hybrid controller for signalised urban grids

This study compares five traffic-signal controllers on a 3×3 SUMO grid and validates the principal findings on a 5×5 grid. The controllers are fixed-time, gap-out actuated, max-pressure, independent tabular Q-learning, and a new hybrid called Pressure-Guided Q-Learning (PGQL). PGQL augments a Q-learner with three actions (hold, follow max-pressure, override max-pressure) and warm-starts the Q-table so the untrained agent initially behaves like max-pressure while learning state-dependent deviations. On the 3×3 grid, the pressure-aware controllers reduced mean waiting time (MWT) by about threefold relative to fixed-time at moderate load. Relative to max-pressure, PGQL was worse at low load, tied near saturation, and better post-saturation. On the 5×5 grid, PGQL outperformed max-pressure at every tested load by up to 6.6%. An action audit showed that learned overrides increased from 28% at low load to 37% under saturation. An ablation replacing max-pressure with gap-out actuation as the warm-start reduced post-saturation performance by about 12%, demonstrating that the choice of warm-start expert matters. Cold-start experiments showed that warm-start primarily improves initialization safety rather than final low-load performance. Plotting MWT against the mean number of vehicles in the network, rather than offered demand, avoids a post-saturation measurement artefact and supports an M/M/1-style carrying-capacity model with a single physically meaningful capacity parameter. The results show that classical adaptive control and reinforcement learning can be combined to achieve measurable post-saturation improvements while retaining max-pressure behavior at initialization, although its formal stability proof does not extend to the learned policy.