Enhancing Energy Efficiency and Economic Benefits with Battery Energy Storage Systems: An Agent-Based Optimization Approach

The emergence of citizen energy communities under the European Clean Energy Package is creating new opportunities for neighboring households to collectively reduce electricity costs through local energy sharing. This paper presents a distributed multi-agent energy management system for a two-household residential energy community in which each household is equipped with photovoltaic generation and a battery energy storage system operating under realistic hourly-varying electricity prices. Each household is managed by an independent Deep Q-Learning agent that learns a cost-optimal charging and discharging policy using only local observations. In parallel, a coordination agent, implemented on the SPADE platform with XMPP-based messaging, oversees real-time peer-to-peer energy transfers between households, enabling energy exchange whenever one household has surplus generation and another faces a deficit. The two households are deliberately configured with complementary profiles: one has higher PV generation capacity while the other has higher energy consumption. This setup creates natural opportunities for local energy sharing between them. Performance is assessed through a three-level evaluation framework: (i) individual household economics (cost reduction, battery management, grid exchanges), (ii) coordination efficiency (transfer frequency, direction, and volume), and (iii) aggregate community performance, which isolates the added value of peer-to-peer sharing beyond what each household achieves through individual BESS optimization. Numerical experiments using GEFCom2014 solar generation data, synthetic residential load profiles calibrated following documented consumption patterns, and day-ahead price signals representative of the Spanish electricity market demonstrate that both Deep Q-Learning agents independently learn effective charge/discharge strategies aligned with price signals and PV availability. They also show that the coordination layer further reduces community grid dependence by routing surplus energy locally rather than exchanging it with the main grid at less favorable rates. The results confirm that a well-engineered integration of decentralized reinforcement learning with a lightweight coordination protocol can deliver measurable economic benefits in realistic residential energy communities without requiring centralized training, shared data, or complex multi-agent reinforcement learning architectures.