Human professional level driving agent for race car simulation environments

Precise vehicle control at the limits of tire adhesion is paramount for both competitive motorsport performance and the safe execution of emergency maneuvers in road vehicles. Mastering this “grip-limit driving” presents significant challenges due to highly non-linear vehicle dynamics and sensitivity to changing conditions, often exceeding the capabilities of traditional controllers and driver models. This paper investigates the efficacy of Deep Reinforcement Learning (DRL), specifically the Proximal Policy Optimisation (PPO) algorithm, as a data-driven approach to learn expert-level driving skills within the TORCS high-fidelity race car simulation environment. An agent was trained end-to-end, utilizing “realworld-friendly” state signals (such as speeds, accelerations, and yaw rate, simple LiDaR, etc.) as input to determine continuous steering and pedal commands. Notably, the trained Agent achieved lap times comparable to a human e-sport world champion on the target track, demonstrating the potential of this methodology while also highlighting how agents can exploit idealized simulation to achieve superhuman control. Furthermore, this work presents the formulation of the time-optimal driving task as a DRL problem and offers a novel justification for the commonly used “progress reward” function, demonstrating its conceptual link to the time-difference feedback mechanisms human drivers use for performance optimization. These findings provide valuable insights into AI-driven vehicle control under extreme conditions and contribute to the development of more capable autonomous agents for simulation and potentially, real-world applications.