ChatGPT as Your Vehicle Co-Pilot: An Initial Attempt

One of the most challenging problems in human-machine co-work is the gap between human intention and the machine's understanding and execution. Large Language Models (LLMs) have been showing superior abilities in solving such issue. In this paper, we design a universal framework that embeds LLMs as a vehicle “Co-Pilot” of driving, which can accomplish specific driving tasks with human intention satisfied based on the information provided. Meanwhile, a utilization workflow is defined to handle the interaction between humans and vehicles, and memory mechanism is introduced to organize the information involved in the tasks. Expert Oriented Black-Box tuning is proposed to improve the performance of the Co-Pilot without finetuning or training the LLMs. In the experiment, the Co-Pilot is applied to two different tasks, i.e., path tracking control and trajectory planning. The Co-Pilot adjusts vehicle operating conditions by selecting a proper controller or planning a certain trajectory to fit human intentions. Simulation tests are conducted to evaluate the performance and generality of the proposed module. The results show that the Co-Pilot can accomplish most of the tasks based on only natural language processing, although it is not flawless. Finally, a discussion about human-machine hybrid intelligence and further applications of LLMs in autonomous driving is made. We believe that such a framework has promising potential in further applications in the field of automous vehicles.