Adaptive energy management of off-grid photovoltaic-proton exchange membrane systems integrated with heat pumps via multi-module reinforcement learning

Operating off-grid photovoltaic-proton exchange membrane electrolyzer (PV-PEM) systems presents significant energy management challenges due to the intermittent nature of solar power. This study investigates an adaptive energy management approach for an off-grid PV-PEM system integrated with heat pump (HP) and battery energy storage (BESS) to maximize surplus solar utilization and reduce hydrogen production costs. To address the computational burden of traditional global optimization, a multi-module Reinforcement Learning-Assisted Energy Management System (RLEMS) is proposed. This innovative architecture effectively decouples long-term strategic threshold optimization from short-term real-time power allocation. The system's performance is evaluated across three distinct climatic regions (Guiyang, Shanghai, and Hami) using different control strategies: fixed-threshold rule-based control (FEMS), FEMS with HP integration (FEMS-HP), and the proposed RLEMS-HP. Results indicate that integrating an HP as a flexible thermal load significantly lowers the levelized cost of hydrogen (LCOH) and improves the solar consumption rate (SCR). Furthermore, the RLEMS-HP strategy consistently outperforms conventional methods by enhancing operational flexibility and robustness against capacity mismatches. The proposed multi-module framework offers a computationally efficient and adaptive solution for off-grid green hydrogen production, achieving an SCR of up to 95.94% and an LCOH as low as 24.81 RMB/kg in high-irradiation regions.