Function approximation reinforcement learning of energy management with the fuzzy REINFORCE for fuel cell hybrid electric vehicles

In the paper, a novel self-learning energy management strategy (EMS) is proposed for fuel cell hybrid electric vehicles (FCHEV) to achieve the hydrogen saving and maintain the battery operation. In the EMS, it is proposed to approximate the EMS policy function with fuzzy inference system (FIS) and learn the policy parameters through policy gradient reinforcement learning (PGRL). Thus, a so-called Fuzzy REINFORCE algorithm is first proposed and studied for EMS problem in the paper. Fuzzy REINFORCE is a model-free method that the EMS agent can learn itself through interactions with environment, which makes it independent of model accuracy, prior knowledge, and expert experience. Meanwhile, to stabilize the training process, a fuzzy baseline function is adopted to approximate the value function based on FIS without affecting the policy gradient direction. Moreover, the drawbacks of traditional reinforcement learning such as high computation burden, long convergence time, can also be overcome. The effectiveness of the proposed methods were verified by Hardware-in-Loop experiments. The adaptability of the proposed method to the changes of driving conditions and system states is also verified.