Reinforcement learning for residential energy storage management at the neighborhood scale: A multi-benchmark evaluation

Electrical grids in warm climates can experience peak strain when afternoon air conditioning demand coincides with residents returning home. Photovoltaic (PV) generation combined with energy storage systems (ESS) can mitigate this by shifting electricity consumption away from peak hours, though controlling ESS for multiple homes is challenging due to varied load and generation profiles. This study presents a simulation-based evaluation of centralized and decentralized Reinforcement Learning (RL) controllers for managing distributed ESS across a 10-house residential neighborhood in Austin, Texas, benchmarking their performance against a mixed-integer linear programming (MILP)-based optimization and a Rule-Based Controller (RBC). The RBC represents schedule-based charging while the MILP performs day-ahead optimization with foresight of building energy loads and PV generation, representing a high-performance benchmark. The decentralized RL controller using the Soft Actor-Critic algorithm achieved 45% electricity cost reductions on average compared to the baseline without storage or control under Time-of-Use pricing, translating to a 51% reduction in peak-period electricity consumption. This substantially outperformed the RBC (20% cost reduction) while delivering competitive performance against the MILP (66% cost reduction). Conversely, the centralized RL controller achieved superior grid-level performance in peak demand and ramping metrics despite lower cost reductions (10%), revealing trade-offs between household-level and grid-level objectives. The RL approach demonstrates practical advantages by using current states, near-term forecasts, and open-source libraries that eliminate costs associated with commercial optimization solvers. This work establishes that RL controllers can effectively manage energy storage across diverse residential neighborhoods, positioning them as viable candidates for deployment in smart grid applications. • Decentralized and centralized RL controllers were benchmarked for ESS management. • The decentralized RL controller achieved 45% cost reductions on average per house. • Decentralized RL controller outperformed the rule-based controller for all metrics. • Centralized RL excelled at grid-level metrics while decentralized RL optimized cost. • RL approach required minimal training data and open-source python libraries.