Frequency Containment Reserve and Imbalance participation: a Battery-Integrated Reinforcement Learning Strategy

With the increasing integration of renewable energy sources (RES), the electrical grid is facing an amplified uncertainty in the energy supply.Transmission System Operators (TSOs) are offering remuneration in exchange for energy exchanges that reduce system imbalances.Helping stabilize the grid frequency is hence an economically viable endeavor, but it requires strategies that can properly manage stochasticities.To tackle this, we analyze the participation of grid-scale batteries in Frequency Containment Reserve (FCR) using a Reinforcement Learning (RL) control strategy.Acting in a multi-market scenario, the RL agent learns to effectively leverage imbalance prices for a high-quality energy recovery strategy.We trained the agent to maximize the imbalance settlement profit while ensuring conforming participation in the FCR service.The agent is also trained to keep the battery yearly cycles below a planned value.In our simulations, we demonstrated the efficacy of RL when dealing with different FCR participation magnitudes, obtaining an average improvement of +9% in profit compared to a rule-based controller baseline.