A Hierarchical Reinforcement Learning Based Bi-Population Optimization Framework for Green Distributed Hybrid Flow-Shop Scheduling with Multiple Crane Transportation

Distributed hybrid flow-shop scheduling problems (DHFSPs) are widely encountered in manufacturing systems. Their complexity increases significantly when multiple overhead cranes are used for material handling. This paper investigates a distributed hybrid flow-shop scheduling problem with multiple overhead crane transportation (DHFSP-MCT), aiming to simultaneously minimize makespan and total energy consumption (including machining and transport). A hierarchical reinforcement learning-based bi-population collaborative metaheuristic algorithm (HRL-BCMA) is proposed. In HRL-BCMA, an iterated greedy strategy is first adopted to generate an initial population. Then, a two-level reinforcement learning framework is designed: a high-level agent decides when to release jobs to the shop floor, while a low-level agent based on a graph isomorphism network selects improvement operators. Furthermore, a bi-population co-evolutionary framework and a knowledge-informed strategy are introduced to enhance solution quality and diversity. Experimental evaluations on both randomly generated instances and a real-world-inspired aluminum manufacturing case show that HRL-BCMA reduces makespan by 8.6% and total energy consumption by 12.3% on average compared to the best existing algorithm (CBMA) while achieving superior Pareto front coverage. These results demonstrate the effectiveness of the proposed method for green scheduling problems with crane transport constraints.