Distributed Deep Reinforcement Learning for Data-Driven Water Heater Model in Smart Grid

The use of clusters of electric water heaters (EWHs) in demand response applications shows great potential. However, the diverse consumption patterns of individual users pose challenges for conventional single-model control strategies. To address this, we propose a distributed reinforcement learning (DRL) strategy for managing EWHs that enhances scalability, adaptability, and user satisfaction. Our approach uses a data-driven model to capture changes in EWH operation and clusters users based on similar consumption behaviors. Offline pre-training is performed for each cluster, followed by individual online training to adapt to user preferences. This distributed framework enables knowledge sharing within clusters, reducing training time and improving scalability over centralized systems. Our method effectively accommodates user variability, balancing system-wide and individual objectives. Experiments demonstrate significant peak demand reductions and enhanced user comfort. Compared to traditional thermostat-based controllers, our method reduces costs by 38%. Additionally, compared to genetic algorithms (GA) and Binary Particle Swarm Optimization (BPSO), our proposed method demonstrates better performance in real-time adaptability and load shifting, providing an efficient solution for demand response management.