Dynamics-aware scheduling of renewable electrolyzer plants: A physics-informed reinforcement learning approach

Producing hydrogen via renewable electrolyzer plants (REPs) is crucial for reducing carbon emissions in industrial sectors. Meanwhile, with renewable energy generation and electrolyzer units integrated at a common grid connection, these plants provide high-quality resources for regulating power grid operations. Nonetheless, fully utilizing this potential relies on the profitable operation of REPs across joint energy and frequency containment reserve markets in conjunction with the application of data-driven reinforcement learning (RL) scheduling approaches to counteract the stochasticity in renewable generation and the volatility of markets. However, current scheduling frameworks often neglect detailed internal thermal-electrical dynamics and discrete operational states, which result in sub-optimal or physically infeasible policies. The present work addresses this issue by proposing a comprehensive imitation-enhanced RL (IE-RL) scheduling framework. First, the framework incorporates a physics-informed neural network surrogate model to capture transient electrolyzer dynamics accurately. Then, the operational environment is decoupled into continuous subspaces based on whether the electrolyzer unit is in the on state or in the standby state, and dedicated expert policies are pre-trained for each operational environment. Finally, the IE-RL agent is trained to imitate the expert policy matching the current active operational state accurately. The results of numerical computations demonstrate that the proposed approach enhances joint-market profitability while maintaining high physical fidelity with a significantly accelerated training process.