Deep reinforcement learning based control for enhanced frequency response with multi-energy storage systems

• GSES-DRL enables adaptive multi-BESS control for fast frequency response. • Genetic algorithm for BESS placement to minimize ROCOF during faults. • Parallel training accelerates DRL convergence and robustness. • Proposed method prevents UFLS relay triggers in severe contingency scenarios. • Validated on various scenarios with superior results over state-of-the-art control. This paper proposes an advanced strategy for managing multiple battery energy storage systems (BESS) to enhance frequency support during contingencies. A novel deep reinforcement learning (DRL) framework based on a guided surrogate-gradient-based evolutionary strategy (GSES) was developed to dynamically regulate BESS outputs for rapid power injection or absorption. This approach effectively mitigates the rate of change of frequency (ROCOF) and stabilizes the system frequency under varying operating conditions. Parallel computing techniques are employed to accelerate training and ensure robust performance. In addition, a genetic algorithm is implemented to determine the placement of BESS within the grid network, strategically minimizing ROCOF during disturbances by accounting for active power injections and inertia contributions from multiple BESS units. The proposed GSES-DRL methodology is rigorously validated using extensive simulations on a modified IEEE 39-bus system, demonstrating effective performance in frequency response compared with existing strategies. In severe scenarios, the proposed method successfully prevents unnecessary load-shedding relay operations, thereby enhancing system stability.