Reinforcement Learning–Based Timetabling of Bus Routes Subject to Oversaturation

Timetables are crucial for the efficient operation of public transportation. This paper tackles the timetabling problem of a regular bus line facing oversaturation due to stochastic fluctuation in demand and a lack of fleet, making it more complex than traditional undersaturated scenarios. By oversaturation, we mean some passengers may not board the first‐arrival bus and must wait for subsequent ones. We propose a reinforcement learning (RL) model based on the Markov decision process (MDP) to optimize the timetable. The objective is to minimize the expected total system costs, involving the costs of passengers and the operator. Constraints concerning vehicle capacity and fleet size are respected. The dynamics of passengers are captured in the state transition process. The proposed model is applied to the No. 8 bus route in Sanya, China. Specifically, the proximal policy optimization (PPO) algorithm is employed. Under deterministic demand, we compare the optimized timetables against the results obtained by classic models. The results show that the proposed model outperforms classical models in terms of both solution quality and computational efficiency. Under stochastic demand, as variance in demand increases, the benefit of the proposed model in the system cost and passenger waiting time becomes more significant. In addition, we optimize the fleet size, accounting for the acquisition cost of vehicles.