A self-learning system for local ramp metering with queue management

This paper presents the design and evaluation process of a self-learning system for local ramp metering control. This system is developed on the basis of reinforcement learning (RL) and can deal with the problem of on-ramp queue management through a continuous learning process. A general framework of the system design including the definition of RL elements and an algorithm that can accomplish the learning process is proposed. Simulation tests are carried out to evaluate the performance of the new system. In terms of the total time spent by road users, the new system can achieve a 30% reduction from the situation of no control, a result which is competitive with the widely accepted algorithm ALINEA. Meanwhile, simulation results show that the new system can keep on-ramp queues strictly under a series of pre-specified constraints, which proves its capability of managing on-ramp queues.