What question did this study set out to answer?

To develop a holistic reinforcement learning framework for whole-building HVAC systems that simultaneously maximizes energy efficiency, maintains thermal comfort, and supports demand response participation.

August 5, 2020Energy and AI242 citationsOpen Access

Reinforcement learning for whole-building HVAC control and demand response

DADonald Azuatalam WLWee-Lih Lee FNFrits de Nijs

Key Points

To develop a holistic reinforcement learning framework for whole-building HVAC systems that simultaneously maximizes energy efficiency, maintains thermal comfort, and supports demand response participation.
Designed an autonomous reinforcement learning controller integrated with a whole-building simulation model.
Trained the agent to adjust HVAC temperature set points dynamically in response to external demand response signals while preserving occupant comfort bounds.
Compared performance against a handcrafted baseline controller and a standard energy-focused reinforcement learning controller.
Standard reinforcement learning operation achieved a maximum weekly energy reduction of up to 22% relative to the handcrafted baseline controller.
The demand response-aware reinforcement learning controller achieved average weekly power increases or reductions of up to 50% compared to the default reinforcement learning controller while maintaining acceptable thermal comfort levels.

Abstract

This paper proposes a novel reinforcement learning (RL) architecture for the efficient scheduling and control of the heating, ventilation and air conditioning (HVAC) system in a commercial building while harnessing its demand response (DR) potentials. With advances in automated building management systems, this can be achieved seamlessly by a smart autonomous RL agent which takes the best action, for example, a change in HVAC temperature set point, necessary to change the electricity usage pattern of a building in response to demand response signals, and with minimal thermal comfort impact to customers. Previous research in this area has tackled only individual aspects of the problem using RL. Specifically, due to the challenges in implementing demand response with whole-building models, simpler analytical models which poorly capture reality have been used instead. And where whole-building models are applied, RL is used for HVAC control mainly to achieve energy efficiency goals while demand response is neglected. Thus, in this research, we implement a holistic framework by designing an efficient RL controller for a whole-building model which learns to optimise and control the HVAC system for improved energy efficiency and thermal comfort levels in addition to achieving demand response goals. Our simulation results show that by applying reinforcement learning for normal HVAC operation, a maximum weekly energy reduction of up to 22% can be achieved compared to a handcrafted baseline controller. Furthermore, by employing a DR-aware RL controller during demand response periods, average power reductions or increases of up to 50% can be achieved on a weekly basis compared to the default RL controller, while keeping occupant thermal comfort levels within acceptable bounds.

Ask AI

Helpful

Bookmark

View Full Paper