Knowledge extraction method for failure identification using multimodal generative AI

Rapid and accurate failure identification is essential for effective equipment maintenance, a key aspect of life cycle engineering (LCE) that supports sustainable manufacturing. Traditional approaches require expert knowledge to define pairs of failure modes and symptoms, which limit scalability and adoption. This study proposes a method using multimodal generative AI to automatically extract failure mode–symptom pairs from troubleshooting flowcharts in equipment manuals. The method recognizes flowchart structure and text, identifies symptoms from decision steps, and links them to failure modes in subsequent action steps. Experiments demonstrate that the proposed approach can efficiently extract these pairs from flowchart images, enabling the automated construction of failure identification models such as Bayesian networks. The results show that the method is nearly ten times faster processing than manual extraction, and for one sample flowchart, 70.4% of the extracted failure mode–symptom pairs matched the correct pairs. Future work will focus on improving extraction accuracy and expanding the evaluation to a broader range of maintenance documents.