Two-stage Q-learning assisted MOEA/D for energy-efficient flexible job shop scheduling considering machine wear

Industry 4.0 prioritizes manufacturing efficiency and energy efficiency, while Industry 5.0 places greater emphasis on the integration of human-centricity, sustainability and resilience. Against this backdrop, equipment health management has become an indispensable consideration in modern manufacturing, as it directly determines the long-term development for manufacturers. Considering this, this paper takes machine wear into account, constructs a novel machine multi-state constrained energy-efficient flexible job shop scheduling problem (MW-MS-EFJSP), which aims to simultaneously minimize makespan, total energy consumption and total machine wear. To solve this problem efficiently, a two-stage Q-learning assisted multi-objective evolutionary algorithm based on decomposition (TSQ-MOEA/D) is proposed, in which Q-learning is employed to assist the selection of local search operators. Specifically, the early stage of Q-learning focuses on pushing individuals toward single-objective boundaries, while the later stage aims to improve the quality of the Pareto front, thereby effectively balancing the global exploration and local exploitation capabilities. To verify the overall performance and practical applicability of the proposed TSQ-MOEA/D, three complementary experimental studies are performed, including ablation analysis, comparative experiments with eight algorithms and industrial case validation. The full experimental results suggest that TSQ-MOEA/D consistently yields high-quality scheduling solutions across various scenarios.