Deep Reinforcement Learning for Airplane Components Failure Prognostics Full Cycle Automation

As airplane components degrade over time, airplane service organizations (e.g., Boeing Global Services) and their airline customers need to collaborate on airplane components failure prognostics and replace/maintain components proactively to improve operation efficiency and reduce cost. In particular, airplane service organizations analyze various sensor data that captures the operational states of airplane components to predict possible component failures. Upon identifying an impending component failure, the service organization promptly sends alerts to the airline maintenance team. In response, the airline maintenance team conducts inspections and maintenance on the component and replaces it if necessary. In this airplane components failure prognostics procedure, machine learning or engineering-based models can be used to make predictions of components failure on each flight. However, it is crucial for airplane service organizations to determine when to send alerts to airlines given the predictions of the full history of flights. Late alerts may cause schedule interruptions or even grounding of the airplane waiting for parts. Early alerts can bring unnecessary inspections that lead to significant cost to airlines. Current solutions rely on heuristics and/or manual engineering reviews to make decisions on sending alerts, which requires significant manual efforts and is difficult to scale. To improve efficiency of airplane components failure prognostics, we applied deep reinforcement learning (RL) to automate the prognostics procedure while enhancing accuracy of alerts timing. Specifically, we used Long Short-Term Memory (LSTM) neural network model to represent alert policy that outputs alerts decisions based on flight sensor data and interaction history with airlines. To train the alert policy, we built a prognostics environment by using probability models to simulate airplane component state transitions over time and the airline’s feedback to alerts. With this environment, the parameters of alert policy are updated to minimize costs for airlines during the simulated prognostics procedure. This is achieved through the Deep Q-Network algorithm with memory prioritization to mitigate reward sparsity issue. Once learned, the alert policy is deployed to make decisions on sending alerts automatically by consuming incoming flight records and parsing current interactions with airlines. Moreover, we can fine-tune alert policy parameters to incorporate new airplane component features and airline operation changes. We conducted a case study on Boeing 787 air cycle machine (ACM) prognostics, which demonstrated the feasibility and effectiveness of our approach.