Comparison of Optimal Control Techniques for Building Energy Management

Optimal controllers can enhance buildings’ energy efficiency by taking forecast and uncertainties into account (e.g., weather and occupancy). This practice results in energy savings by making better use of energy systems within the buildings. Even though the benefits of advanced optimal controllers have been demonstrated in several research studies and some demonstration cases, the adoption of these techniques in the built environment remains somewhat limited. One of the main reasons is that these novel control algorithms continue to be evaluated individually. This hampers the identification of best practices to deploy optimal control widely in the building sector. This paper implements and compares variations of model predictive control (MPC), reinforcement learning (RL), and reinforced model predictive control (RL-MPC) in the same optimal control problem for building energy management. Particularly, variations of the controllers’ hyperparameters like the control step, the prediction horizon, the state-action spaces, the learning algorithm, or the network architecture of the value function are investigated. The building optimization testing (BOPTEST) framework is used as the simulation benchmark to carry out the study as it offers standardized testing scenarios. The results reveal that, contrary to what is stated in previous literature, model-free RL approaches poorly perform when tested in building environments with realistic system dynamics. Even when a model is available and simulation-based RL can be implemented, MPC outperforms RL for an equivalent formulation of the optimal control problem. The performance gap between both controllers reduces when using the RL-MPC algorithm that merges elements from both families of methods.