Scalable reinforcement learning approach for scheduling customer orders with missing operations

This study addresses the customer order scheduling problem with missing operations, a complex combinatorial optimisation task with significant industrial relevance, aiming to minimise total tardiness. We propose an RL-based algorithm for solving this problem enhanced with curriculum learning to achieve strong scalability and generalisation. The model employs an encoder–decoder architecture that embeds missing operations, processing times, and due dates, and sequentially generates schedules in an autoregressive manner. A preference-based loss function is integrated into the training objective to align policy updates with performance ranking, further improving solution quality and convergence. An optional local search procedure can additionally be applied as a post-processing step when additional computation time is available. Experimental results on 640 test instances show that Proposed consistently outperforms traditional heuristics, state-of-the-art metaheuristics, and a recent learning-based baseline in solution quality, while Proposed+LS provides further improvement when additional computation time is available. Proposed also demonstrates strong scalability, performing effectively across job sizes from 100 to 300 and machine sizes from 5 to 10. These findings demonstrate the effectiveness of curriculum-based RL on the considered benchmark instances and provide a basis for its further evaluation in practical industrial scheduling environments.