With the rapid development of smart energy systems, including smart grids, integrated energy networks, and edge-enabled industrial platforms, the complexity of scheduling and coordination has significantly increased. A key challenge arises from the coexistence of multiple operational layers and heterogeneous time scales, which makes joint optimization and policy alignment difficult due to asynchronous updates and nonuniform temporal dynamics. To address this issue, we propose a hierarchical scheduling framework based on federated reinforcement learning, which enables distributed agents to independently learn local scheduling strategies while periodically sharing abstract behavioral representations with a central coordinator. The framework incorporates a multi-level pattern encoder that extracts temporal and structural decision features from local trajectories, which are aggregated to construct a global behavioral prior for guiding local updates. This design enhances local adaptability while maintaining global consistency, and ensures privacy by avoiding raw data exchange. We validate the proposed method in three representative scenarios — multi-scale smart grid dispatch, industrial manufacturing scheduling, and edge energy management — where agents operate under distinct temporal and structural constraints. Experimental results demonstrate that our approach achieves faster convergence, better policy generalization, and higher stability compared with baseline methods under asynchronous conditions. Furthermore, visualization analyses confirm that the framework facilitates interpretable policy clustering and preserves robustness across heterogeneous system configurations. Overall, this study provides an effective, scalable, and privacy-preserving solution for intelligent energy system scheduling under multi-level optimization and time-scale coordination.
Zhao et al. (2025) studied this question.