Route Planning and Power Management for PHEVs With Reinforcement Learning

In this article, we propose a novel control scheme for route planning with power management for plug-in hybrid electric vehicles (PHEVs). By considering the power management of PHEVs, we aim to find the route that leads to the minimum energy consumption. The scheme adopts a two-loop structure to achieve the control objective. Specifically, in the outer loop, the minimum energy consumption route is obtained by minimizing the difference between the value function of current round and the best value from all previous rounds. In the inner loop, the energy consumption index with respect to PHEV power management for each feasible route is trained with reinforcement learning (RL). Under the RL framework, a nonlinear approximator structure, which consists of an actor approximator and a critic approximator, is built to approximate control actions and energy consumption. In addition, the convergence of value function for PHEV power management in the inner loop and asymptotical stability of the closed-loop system are rigorously studied. Using Toyota Prius as the vehicle model and planning its route in a city with various navigation ranges, we demonstrate the effectiveness of the proposed control scheme with simulation studies.