Deep reinforcement learning for adaptive flexible job shop scheduling: coping with variability and uncertainty

The frequently evolving manufacturing system necessitates real-time large-scale data analytics, surpassing the capabilities of classical systems or human skills, necessitating knowledge development and self-adaptive control. Unpredictable real-time events in smart factories cause changes in the effectiveness of planned schedules or tasks; even slight interruptions can accumulate to make the preplanned schedule unoptimized, if not impossible. To meet the required production efficiency, typical dynamic scenarios on the shop floor, such as failures, random job arrivals, and machine setup, must be addressed quickly. Hence, this study proposes a novel Triple Deep Q Network (TDQN) approach for learning high-quality dispatching rules for addressing the Flexible Job Shop Scheduling Problem (FJSSP) in uncertainty. For the dispatching rule-based FJSSP, the Markov Decision Process (MDP) to choose a suitable operation-machine (O-M) pair is formulated to allow operation selection and resource assignment resolutions to be made simultaneously. The performance of the TDQN method is investigated against the Double Deep Q Network (DDQN) and Deep Q Network (DQN) approach and found to be more stable. Moreover, among the 29 different shopfloor configuration settings, TDQN performed better in 62.07% of the overall instances, while DDQN and DQN performed better in only 31.03% and 6.9% of instances, respectively.