Online Scheduling for Integrated Process Planning and Scheduling Problem Based on Proximal Policy Optimization

This is the authors' abstract. We don't add key points for this paper.

Integrated optimization of process planning and shop scheduling is pivotal to enhancing the overall efficiency of manufacturing systems. To address dynamic disturbances in integrated process planning and scheduling (DIPPS) problems, this paper proposes an online scheduling method leveraging proximal policy optimization (PPO). Specifically, an OR-graph description matrix is devised to encode job-related information; an online scheduling framework is also developed to identify decision-making nodes and operational information. A PPO-trained intelligent agent is further deployed to execute real-time decision-making. For the agent, state features and composite rules are defined as its state and action spaces, respectively. Furthermore, a reward function integrating both local and global reward mechanisms is proposed. The proposed algorithm is validated on test cases with varying scales and dynamic disturbance frequencies. Compared with conventional scheduling rules, the method's efficiency is verified, and its practical applicability is demonstrated via a real-world engineering case.