Weather-adaptive reinforcement learning framework for heat pump control under time-of-use tariffs

Controllers for heat pumps (HPs) in space heating are essential to reduce electricity costs while maintaining thermal comfort for customers billed under time-of-use (TOU) pricing, enabling HPs to act as flexible loads in demand-side management. Although reinforcement learning (RL) can learn cost-optimal control policies, its performance often degrades across outdoor temperatures. To address this limitation, we propose a mixture-of-experts (MoE) framework that trains advantage actor-critic (A2C) RL agents on representative temperature profiles and selects the appropriate agent using temperature forecasts. The proposed framework consistently reduces electricity costs across outdoor temperatures from 10 to −18 °C, achieving an average reduction of 16.5% relative to a well-tuned rule-based controller (RBC) under cold test scenarios, compared to 10.1% for the best single-policy A2C, while also demonstrating greater stability across random seeds. Under colder conditions, the proposed A2C MoE achieves cost reductions of 21.3% and 22.4% relative to the RBC across two datasets, while single-agent A2C baselines show inconsistent performance and may underperform the RBC. The results demonstrate that the proposed framework improves both cost reduction and generalization to colder outdoor temperatures.