Sequential multi-objective multi-agent reinforcement learning approach for system predictive maintenance of turbofan engine

Existing predictive maintenance (PdM) methods typically focus solely on whether to replace system components without considering the costs incurred by inspection. However, a well-considered approach should be able to minimize Remaining Useful Life (RUL) at engine replacement while maximizing inspection interval. To achieve this, multi-agent reinforcement learning (MARL) can be introduced. However, due to the sequential and mutually constraining nature of these 2 objectives, conventional MARL is not applicable. Therefore, this paper introduces a novel framework and develops a Sequential Multi-objective Multi-agent Proximal Policy Optimization (SMOMA-PPO) algorithm. Furthermore, to provide comprehensive and effective degradation information to RL agents, we also employed Gated Recurrent Unit, Quantile Regression (QR), and probability distribution fitting to develop a GRU-based RUL Prediction (GRP) model. Experiments demonstrate that the GRP method significantly improves the accuracy of RUL predictions in the later stages of system operation compared to existing methods. When incorporating its output into SMOMA-PPO, we achieve at least a 15 % reduction in average RUL without unscheduled replacements (UR), nearly a 10 % increase in inspection interval, and an overall decrease in maintenance costs. In industries like aerospace and manufacturing, where downtime and costs must be minimized without sacrificing safety, our approach optimizes RUL and inspection intervals through multi-objective maintenance planning. It boosts reliability, efficiency, and safety while cutting expenses, making it a valuable tool for real-world PdM.