Knowledge-Augmented Large Language Models to support Automated HAZOP Report Generation

The Hazard and Operability (HAZOP) study is a widely used method for hazard identification and analysis in process industries. However, its preparation remains highly time-consuming and resource intensive. Recent advances in large language models (LLMs) offer new opportunities for automating HAZOP worksheet generation. Yet concerns persist regarding hallucinations associated with LLMs, low valid scenario ratios, and strong bias toward administrative safeguards. To address these challenges, this study presents a domain augmented LLM method that aims at enhancing the capability of LLMs in generating HAZOP worksheets. First, multimodal LLMs are employed to generate node descriptions from piping and instrumentation diagrams (P&ID) node images. Second, a domain-adaptive pretraining model is developed to enhance hazard-specific semantic representations. Third, a retrieval-augmented generation (RAG) model is conducted to ground LLMs (GPT-4o, GPT-4o-mini, and LlaMa 3.2) in external HAZOP-related knowledge. A total of 6,120 past HAZOP records and 1,140 incident reports were structured in JavaScript Object Notation (JSON) format, while embeddings of P&ID node descriptions and incident narratives were indexed using Facebook AI Similarity Search (FAISS) to enable similarity-based retrieval. Retrieved knowledge is combined with the P&ID node and its generated description into a role-based chain of thought (CoT) prompt to guide LLMs reasoning. The proposed method was evaluated on a reference P&ID node case and further validated on two additional nodes through an ablation study. The findings demonstrate that RAG-based models incorporating domain-adapted embeddings achieve higher semantic similarity (F1-scores > 92%) compared to 82–87% for non-grounded LLMs. The valid scenario ratio improves by approximately 10–18%, with RAG + GPT-4o + domain-adapted embedding + role-based CoT prompting achieving the highest ratio of 48.2%. In addition, deviation identification is enhanced, reaching a maximum valid ratio of 81.25%. Retrieval grounding further reduces the over-generation of procedural safeguards and increases the proportion of active and passive safeguards, producing distributions closer to the reference worksheet compared to non-grounded models. These findings confirm the effectiveness of domain augmented LLMs for supporting automated HAZOP worksheet generation.