Dynamic Jobshop Scheduling Algorithm Based on Deep Q Network

Jobshop scheduling is a classic instance in the field of production scheduling. Solving and optimizing the scheduling process of the jobshop can greatly reduce the production cost of the workshop and improve the processing efficiency, thereby improving the market competitiveness of the manufacturing industry. A reasonable scheduling algorithm in jobshop can reduce manufacturing costs, shorten product delivery time, enhance customer service quality, and improve the overall performance of the manufacturing system. The jobshop scheduling problem is transformed into a reinforcement learning problem based on the Markov decision process. The performance of the adaptive scheduling algorithm in a dynamic manufacturing environment is improved based on the Deep Q Network (DQN). In the proposed scheduling algorithm, five state features of continuous value ranges are designed for input to a Deep Neural Network (DNN), as well as ten well-known heuristic dispatching rules are selected as the action set of the DQN. In the proposed scheduling algorithm, the target network and the prediction network are used to train the parameters. The DQN uses a “softmax” function as an action selection strategy during the training process, where it selects the dispatching rules with the greatest action value as the execution action. It resolves the problem whereby the sub-optimal action value is greater than the Q value of the optimal action in the early learning stage. Futhermore, the non-optimal action is selected with a greater probability in the later learning stage. Ten typical jobshop test instances called “LA” are used as a simulation object and operated in Python. The simulation results confirm that the proposed scheduling algorithm based on DQN has better performance and universality than a single dispatching rule or traditional Q learning algorithm.