Improving Energy Flexibility in Photovoltaic-Battery Systems through Switching Reinforcement Learning Control

The electric power sector is undergoing radical changes to enhance sustainability. A main part of this transformation involves the integration of more renewable energy sources (RES) into the power grid, emphasizing the critical need for increased grid flexibility. However, current power grid controllers face challenges in achieving this flexibility due to the unpredictable nature of RES and load, compounded by the complex multi-physical nature of the entire energy system. To tackle these challenges and improve existing controllers, this paper introduces a novel control strategy employing Reinforcement Learning (RL). The proposed strategy is validated through its application to a Photovoltaic (PV)-battery system, where the RL agent learns through interactions with the environment, guided by a reward function based on energy costs. Recognizing the difficulty of maintaining optimality in dynamic environments with single-model RL, our approach advocates for a multi-model RL framework. The framework incorporates a clustering algorithm to identify the optimal number of models. Additionally, we introduce a switching RL strategy involving an offline phase for dataset partitioning and RL model training, followed by an online phase where a real-time switching decision unit selects the optimal RL model, dynamically adapting to current operating conditions. Applying our proposed approach to the PV-battery system demonstrates an 8.15% increase in cost efficiency compared to a conventional single-model RL framework.