District heating optimization in residential buildings using reinforcement learning with adaptive context-aware predictive environment

As district heating networks evolve to meet climate-neutral objectives, optimizing their control under heterogeneous building characteristics and dynamic environmental conditions remains a significant challenge. Traditional control strategies often lack the adaptability necessary to account for building-specific dynamics and to ensure real-time adherence to operational safety constraints. In this work, we present an integrated machine learning-based framework that combines an adaptive context-aware transformer model with deep reinforcement learning to address these limitations. The proposed approach introduces an adaptive context-aware transformer as a predictive environment within a Deep Q-Network (DQN) framework, enabling data-driven, building-specific control of district heating systems. Utilizing real-world data from 148 residential buildings across Sweden and Finland, the model incorporates contextual embeddings and temporal features to predict indoor temperature trajectories with high accuracy, achieving root mean square error values between 0.18–0.24 °C for Swedish buildings and 0.26–0.32 °C for Finnish buildings. The DQN agent leverages these predictions to optimize heating control while ensuring compliance with operational safety limits and preserving occupant comfort. Experimental results demonstrate significant energy savings, with mid-rise buildings achieving up to 14.85% reduction in energy consumption, and peak seasonal savings exceeding 20% during spring months. This integrated approach illustrates the potential for substantial energy optimization and reliable indoor climate management in future district heating networks.