Reinforcement learning-based control algorithm for connected and automated vehicles on a two-lane highway section with a moving bottleneck

To address the problem of control optimization for connected and automated vehicles (CAVs) on a highway section with two lanes in the same direction (hereafter referred to as a two-lane highway section) under a moving bottleneck scenario and to improve their traffic efficiency, this study proposes a control model based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. The model fully incorporates the state information of the ego vehicle and surrounding vehicles as input to generate continuous control actions as output, and utilizes a multi-objective reward function to guide the CAV’s behavior comprehensively. To further enhance training efficiency and driving safety, a collision avoidance strategy based on the inverse Time-to-Collision criterion is integrated into the framework, which effectively mitigates the risk of collision caused by inappropriate acceleration decisions. In the simulation experiments, the proposed method is evaluated under various CAV entering probabilities and compared against a conventional heuristic rules-based algorithm (HRA). The results demonstrate that the TD3-based control strategy outperforms the HRA across multiple performance metrics. In particular, under high-density conditions, the overtaking throughput is improved by approximately 40%, significantly enhancing traffic efficiency in the presence of moving bottlenecks.