Evolution Strategies-Guided Deep Reinforcement Learning for Dynamic Hybrid Flow-Shop Scheduling Problem

Flexible manufacturing faces the challenge of increasing productivity and conserving resources, especially in complex production environments with dynamic event. This paper addresses a dynamic Hybrid Flow-shop Scheduling Problem (HFSP) with unrelated parallel machines using a Deep Reinforcement Learning (DRL) approach to intelligently allocate continuous new job arrivals while minimizing the total weighted tardiness cost. In this paper, Evolution Strategies-guided Deep Reinforcement Learning (ES-DRL) scheduling model is proposed by designing appropriate state features, scheduling actions, and training strategies. In addition, goal-directed composite rules are proposed to provide effective scheduling actions. Meanwhile, the state transition in the environment is adjusted by introducing key state. The ES-DRL model is then trained to make decisions, indicating the reasoning behind the system design. Experimental results show that ES-DRL outperforms the other comparison algorithms regarding significance. In addition, the experiments are extended to the multi-factories system to further validate the scalability and adaptability of the scheduling model, and this extension also yields encouraging results. These results affirm the universal applicability of ES-DRL for dynamic HFSP.