ABSTRACT This paper presents an integrated hybrid reinforcement learning–model predictive control (RLMPC) framework for autonomous highway systems, unifying macroscopic traffic flow regulation and microscopic on‐ramp merging control. At the macroscopic level, a ramp metering (RM) controller based on a data‐driven model predictive control (MPC) formulation using second‐order Q‐learning is implemented in the METANET environment on a benchmark three‐segment freeway without the need for explicit traffic models. The RLMPC RM learns optimal flow regulation directly from closed‐loop data, achieving enhanced system performance, constraint satisfaction and smooth control compared to common RM algorithms such as ALINEA, MPC and deep RL. At the microscopic level, an RLMPC merging controller manages autonomous on‐ramp manoeuvres in which an ego vehicle enters the mainline approximately 160 m before the merge point and completes the manoeuvre 50 m downstream while interacting with surrounding vehicles. In this phase, when a collision risk arises, the MPC takes control; otherwise, the reinforcement learning (RL) policy operates, combining model‐based safety with learning‐based efficiency and yielding superior overall performance. Evaluations under varied traffic conditions show that implementing RM at the macroscopic level significantly improves microscopic on‐ramp merging performance. Relative to the no‐RM baseline, the framework achieves a 34.5% reduction in merge time under slow traffic conditions, eliminates collision events and moderately enhances overall efficiency and driving comfort.
Pourghavam et al. (Thu,) studied this question.