Market competition is increasingly intense and sustainable development has attracted widespread attention. The flexible job shop scheduling problem requires the collaborative optimization of production efficiency and machine energy consumption. This scheduling problem has high solution complexity. It is difficult to balance multiple conflicting objectives and obtain stable scheduling results with traditional optimization methods. A Dual-Layer Proximal Policy Optimization algorithm (DL-PPO) based on a hierarchical decision-making mechanism is proposed to achieve the collaborative optimization of production efficiency and energy consumption in solving the Energy-Aware Flexible Job Shop Scheduling Problem (EA-FJSP). First, a hierarchical scheduling framework based on DL-PPO is designed to solve the EA-FJSP. In this framework, the high-level controller selects sub-objectives from a global optimization perspective, while the low-level controller executes feasible dispatching rules according to the selected sub-objectives. Twelve key state features extracted from four dimensions, time, energy consumption, job, and machine, are used to construct a multi-dimensional state space. These features enable a comprehensive state representation of the scheduling environment and provide accurate input for the DL-PPO. The global optimization objective is decomposed into four sub-objectives employing a goal decoupling policy. Four dedicated reward functions are designed for the sub-objectives to guide the low-level controller to make optimal decisions in terms of time and energy consumption, thereby achieving multi-objective collaborative optimization. Considering the two decisions of job selection and machine assignment in solving the EA-FJSP, twenty dual-decision-point dispatching rules are designed as the action space for the low-level controller to achieve the global optimization objective. Finally, the effectiveness, applicability, and superiority of the DL-PPO in EA-FJSP are demonstrated through comparisons with dispatching rules and other deep reinforcement learning methods.
Qiao et al. (Mon,) studied this question.