Deep reinforcement learning for citizen-centric building retrofit decision making under behavioral uncertainty of a series game scenario

Residential buildings account for a significant portion of global energy consumption, and energy retrofits represent a crucial pathway toward achieving carbon neutrality. However, homeowners often make retrofit decisions under bounded rationality, influenced by social factors, limited information, and random opportunities rather than optimal economic calculations. This study proposes a novel framework that models citizens' inherent decision-making randomness through a serious game environment (Net-Zero Home Race Game, NZHR) integrated with EnergyPlus building simulation and derives optimal retrofit strategies using Proximal Policy Optimization (PPO), a deep reinforcement learning algorithm. The NZHR game uses a representative single-family residential building model from the NREL ResStock database located in the Dallas-Fort Worth metropolitan area of Texas, where a player encounters stochastic retrofit opportunities, weather events, and occupant behavior changes over a 20-year simulated horizon. The retrofit decision-making is formulated as a Markov Decision Process (MDP) with a state space encompassing capital, retrofit portfolio, weather, and occupant behavior variables, and a binary action space (purchase or decline). The PPO agent was trained for 500 episodes and evaluated over 30 test episodes. The trained PPO agent achieved mean energy savings of 12,692 kWh (26.4% reduction) and final capital of $49,647. The agent learned a phased investment strategy: prioritizing envelope insulation in early years (Years 1-7), followed by HVAC improvements mid-horizon (Years 8-12), and deferring high-cost solar PV installation until late years (Years 13-20) when sufficient capital had accumulated. This research bridges the gap between idealized optimization models and real-world behavioral uncertainty in retrofit decision-making.