Race Strategy Reinforcement Learning: Optimising Pitstop Strategy with Emergent Tactics in Formula One

In Formula One, often described as the pinnacle of motorsport, teams compete to design and produce the fastest cars, driven by some of the best drivers in the world, in order to win races. However, a team has little chance of success without effective race strategy , i.e. selecting which tyre compounds to use and when to take pitstops to change between them. Teams’ methods for solving this problem are usually limited to linear optimisation, while some run Monte Carlo simulations in simple, best-case situations; these approaches thus fail to take into account the complex interactions between teams’ strategies and tactics in this unpredictable multi-agent environment. Further, there is low uptake of AI in this domain, potentially due to a lack of trust in these “black-box” models. In this work, we enable the massive potential of reinforcement learning (RL) models in this space using post-hoc techniques from explainable AI. Specifically, we introduce Race Strategy Reinforcement Learning (RSRL), an RL model which allows us to control the strategies of cars in race simulations, with explanations for their actions to help foster trust in users. We first demonstrate that RSRL outperforms baselines of hard-coded and Monte-Carlo strategies, presenting opportunities for improving race strategy for all Formula One teams, and potentially beyond, especially in other areas of motorsport. Next, we analyse RSRL’s generalisability to unseen tracks and show how performance on one or multiple tracks can be prioritised via training. We then exhibit the fidelity and comprehensibility of the deployed explanations towards improving user trust in RSRL’s decisions. Finally, we highlight the emergent tactics , i.e. emergent behaviours representing real-world tactics, learnt by RSRL, pointing towards the general applicability of RL for modelling and even influencing race strategy in Formula One.