End-to-End Imitating Personalized Lane Change Policy from a Few Demonstrations

Personalized policy imitation of autonomous vehicles (AVs) is an important technique to lessen human–machine conflict and improve AVs’ predictability to other traffic participants. However, existing imitation learning methods, both model-free and model-based, fail to solve this issue effectively and efficiently if only one demonstration of the target driver is available. We propose a personalized lane change policy for autonomous driving, which can end-to-end learn human drivers’ behavior from a few demonstrations. To this end, combining model-free imitation learning and a model-based differentiable optimization method, we establish a policy generator with learnable personalized parameters in the cost function. By differentiating the equations that satisfy Pontryagin’s Maximum Principle (PMP), we solve the gradient of imitation loss with respect to learnable personalized parameters. The gradient is used to update and ultimately recover these personalized parameters in an end-to-end learning fashion. Here, “end-to-end” means that the personalized parameters are recovered by gradient-based optimization inside a structured model-based control framework, rather than by learning direct observation-to-action mapping. With these recovered parameters, the proposed policy generator imitates drivers’ behavior. The results prove the following: (1) the proposed method can mimic different drivers’ lane change policies with both sufficient data and only one demonstration by recovering the personalized cost function automatically, and (2) by recovering the personalized cost function automatically, the proposed method achieves a comparable final imitation loss to the differentiable-MPC baseline, with faster and more stable training convergence and a substantially lower computational cost in the one-demonstration setting.