Optimal scheduling of hydrogen-integrated hybrid energy storage systems via multi-agent coordinated reinforcement learning

Hydrogen-integrated hybrid energy storage systems (HI-HESS) provide an effective approach for improving renewable energy accommodation and multi-energy coordinated operation. However, the dynamic coupling among electricity, heat, and hydrogen energy carriers, as well as the coordination challenges among multiple agents, significantly increase the difficulty of optimal scheduling. To address this issue, this paper proposes a multi-agent optimal compromise proximal policy optimization (MAOCPPO) method for the optimal dispatch of HI-HESS. First, a multi-agent optimal dispatch model is established with the objective of minimizing the overall operating cost of the integrated energy system. Then, a multi-agent reinforcement learning framework consisting of HIEP-Agent, ELA-Agent, TLA-Agent, and HLA-Agent is constructed. To improve coordination among heterogeneous agents, an explicit coordination layer with dependency-aware coordinated action generation is introduced to construct multiple candidate coordination configurations. Furthermore, a compromise-oriented joint decision selection mechanism based on TOPSIS is employed to identify a balanced scheduling solution from the generated candidates. Simulation results demonstrate that the proposed method achieves lower operating cost than PSO, MADDPG, and MAPPO, while exhibiting faster and more stable convergence than the reinforcement learning baselines. Coordination baseline and ablation analyses, together with comparisons of different compromise-selection methods and sensitivity analyses of criterion weights and normalization schemes, further demonstrate the effectiveness of the proposed coordination mechanism. In addition, robustness analyses under renewable energy uncertainty and different hydrogen storage capacity configurations verify the adaptability of the proposed method under varying operating conditions. The proposed MAOCPPO method effectively improves the economic performance and coordinated operation capability of HI-HESS.