Large Language Model Guided Reinforcement Learning for Heat Pump Control in Smart Buildings

Heat pumps are essential for the decarbonization of the heating sector but often operate inefficiently due to volatile electricity prices, forecast uncertainties, and diverse comfort preferences. Traditional rule- and model-based controllers ensure stable operation but lack adaptability under dynamic conditions, while data-driven reinforcement learning (RL) provides decision-making flexibility at the cost of interpretability and transparency. This paper presents a hybrid high-level control framework in which a large language model (LLM) serves as a meta-policy, supervising a portfolio of RL-based low-level controllers for residential heat pump operation, formulated as a partially observable Markov decision process (POMDP). The LLM processes structured JSON-formatted prompts containing building states, short-horizon forecasts, candidate actions, and one-step rewards, and outputs either a discrete controller selection (guided mode) or convex weights for action fusion (weighted mode). Benchmark experiments in a realistic building simulation testbed compare four state-of-the-art LLMs (Mistral-7B, Llama-3.1-8B, DeepSeek-Chat, and GPT-4.1) against standalone RL (PPO, SAC, DDPG, and A2C). A two-dimensional decoding sensitivity analysis identifies balanced stochastic decoding hyperparameter regions (temperature and nucleus sampling) that prevent response errors, thereby stabilizing control performance. Results show that LLM-guided RL achieves up to 5.1\% higher mean reward over the best standalone RL controller.