Experimental demonstration of time-efficient auto-Calibration of a vehicle thermal management system using safe reinforcement learning

• Implementation of safe and time-efficient exploration in Reinforcement Learning-based control of an automotive system. • Experimental validation demonstrating robustness under varying ambient conditions within a vehicle climatic chamber. • Development of a model-free Reinforcement Learning-based control approach requiring minimal a-priori system knowledge. • Safe exploration realized by limiting the agent’s action step-size using reciprocal control barrier function when approaching the safety boundary. • Information gain metrics defined based on action-value estimates from the Gaussian Process Regression model used for time-efficient exploration. Future automotive powertrains become increasingly complex to achieve optimal and robust performance in real-world conditions. This is accompanied by rapidly escalating time and cost demands for calibrating powertrain control systems using existing map- and model-based methods. This challenge highlights the urgent need for self learning control strategies, which autonomously learn optimal control settings on the road. This work demonstrates the potential of Reinforcement Learning-based self learning control for a battery electric vehicle thermal system with safety constraints. To realize Reinforcement Learning on the tested vehicle, a novel exploration method is implemented, which explicitly deals with system safety and minimizes experiment time. By combining an online-learned Gaussian Progress Regression model and a reciprocal Control Barrier Function, the optimal direction and step size for information-rich actions is determined during exploration. Validated on a vehicle test-bench, the proposed method calibrates the reference generator to optimize steady-state operation of the heat pump system. Safe and robust performance is demonstrated for varying ambient temperature and humidity, achieving a relative error in heat pump efficiency within 2% of the true optimum across all validation points. Compared to conventional map-based control, the Reinforcement Learning-based approach reduces calibration time by 69%.