Ecological Adaptive Cruise Control for Vehicles With Step-Gear Transmission Based on Reinforcement Learning

In this paper an ecological adaptive cruise controller to reduce the fuel consumption and ensure the safe inter-vehicle distance for vehicles with step-gear transmissions is presented. An optimal control strategy using reinforcement learning with a novel actor-gear-critic architecture is proposed to obtain the continuous traction force trajectory and the discrete gear shift schedule. The traction force is determined from an actor network to maintain a desired inter-vehicle distance which improves the driving safety in a car-following process. The gear shift schedule is derived from a gear network to reduce the fuel consumption. The control strategy is model-free and allows continuous online learning for different driving situations without look-ahead velocity information. Particularly the nonlinear vehicle dynamics, the nonlinear transmission efficiency map for different gear ratios, and the nonlinear fuel consumption map are learned for fuel consumption reduction. The proposed controller is evaluated for different driving scenarios to demonstrate its robustness. Furthermore simulation comparisons for different gear shift schedules and velocity trajectories are given underling the advantages in terms of fuel economy and driving safety.