Safe mean-field reinforcement learning for cooperative control of large-scale multi-zone HVAC systems

Efficient control of HVAC systems is essential for reducing building energy consumption while maintaining indoor environmental quality. However, in multi-zone scenarios, strong thermal coupling among zones, together with interactions among temperature, CO 2 concentration, and humidity, results in highly nonlinear and strongly coupled dynamics, thereby significantly increasing the complexity of control decisions. Furthermore, as the number of zones grows, the joint state–action space expands rapidly, leading to the curse of dimensionality and difficulties in coordinated optimization. Consequently, traditional methods struggle to achieve both scalability and satisfactory control performance. To address these challenges, we develop a dynamic optimization model for multi-zone HVAC systems, where multi-variable coupled control is formulated as a multi-agent decision-making problem, capturing both inter-zone and inter-variable interactions. Building upon this, we propose a Safe Mean-Field TD3 (SMF-TD3) algorithm to enable efficient and scalable cooperative control. Specifically, mean-field theory is introduced to approximate individual interactions via aggregated behaviors, thereby reducing system complexity and enabling scalable decision-making. Meanwhile, to satisfy thermal comfort, air quality, and equipment safety constraints, a Lagrangian-based safe reinforcement learning mechanism is adopted, where constraints are embedded into policy optimization and Lagrange multipliers are adaptively updated to balance performance and constraint satisfaction. Finally, in the seven-day, 12-zone simulation, SMF-TD3 reduces electricity cost by 3.37% relative to CPO and 17.11% relative to MADDPG, and cumulative constraint violations by 75.05% relative to MF-TD3. Across the 12-, 20-, and 30-zone systems, training time is reduced by 52.48%–58.44% relative to MASAC, supporting improved economic performance, constraint satisfaction, and scalability under the tested conditions.