Reinforcement Learning for Residential Heat Pump Operation

Residential space heating accounted for approximately 19% of the overall energy consumption of Germany in 2021. Therefore, the efficient operation of electrified heating systems is of major importance for the energy transition. We apply a reinforcement learning approach for operating a district heat pump and compare results with a classic rule-based approach. No building model is required in our study but only basic parameters of the hot water tank along with demand and ambient temperature data which all is easily attainable. Additionally, the environment is designed in a way that the residents living comfort is never compromised which maximizes applicability in real world buildings. The agent is able to exploit variable electricity prices and the flexibility of the hot water tank in such a way, that up to 35% of energy costs could be saved. Additionally, depending on the agent's settings, only 23% to 41% of the heat pump's nominal power installed according to current standards was used. The robustness of the approach is shown by running ten independent training and testing cycles for all setups with reproducible results. The importance of demand forecasts is evaluated by testing different observation spaces of the RL agent. Even if the agent has no demand information at all, costs savings still are 25%.