Dynamic control of schedule-based and headway-based bus services using deep reinforcement learning

Operational deviations caused by stochastic demand and traffic conditions, as well as unexpected breakdowns, can degrade the reliability and efficiency of transit operations. This study proposes frameworks utilising a Double Deep Q-Network architecture to optimise real-time bus control. We compare DRL agents against a ‘no-control’ and a rule-based strategy across two distinct operational environments: schedule-based and headway-based services, under both regular and severely disrupted conditions. The proposed framework evaluates a hierarchy of control actions, ranging from holding-only to integrated holding, stop-skipping, and speed adjustment. Results indicate that in regular operations, rule-based holding-only strategies yield inefficient outcomes in terms of total journey times. In contrast, the DRL model integrating holding, stop-skipping, and speed adjustment achieves significant improvement, reducing the total passenger journey time median up to 34% and limiting journey time variability. Under severe disruptions, DRL agents in low-frequency scenarios effectively employ stop-skipping to limit delay propagation, which can result in 24% median total journey time reduction and balance passenger loads, while in high-frequency scenarios, the integrated control prevents bus bunching. This study demonstrates that equipping control systems with different tactics yields a more robust, equitable, and resilient transit service as the agent can adapt its performance to different stochasticity levels and delay conditions.