Prompt Engineering to Classify Components of Standard Operating Procedure Steps Using Large Language Model (LLM)-Based Chatbots

Mission safety and efficiency is achieved when flight crews can complete Standard Operating Procedures (SOPs) within the Available Operational Time Window (AOTW). This is especially the case for abnormal and emergency procedures. To ensure SOPs can be completed within the AOTW, SOP Designers use Monte Carlo simulations to model the SOP performance. These simulations necessitate the decomposition of an SOP step into its constituent components, a process traditionally reliant on labor-intensive manual classification. Recent advancements in Large Language Models (LLMs) and the accompanying automation facilitated by Prompt Engineering (PE) present promising avenues for streamlining the classification of SOP steps, thereby potentially expediting the SOP design process.This paper describes a method for Prompt Engineering (PE) aimed at the automation of component classification within Large Language Model (LLM)-enabled chatbots. The classified components can be assigned time distributions for a Monte Carlo simulation to evaluate SOP performance. A case study is described in which the LLM correctly classified 80% of SOP step components, saving 36% of the time to conduct the analysis. The implications of the method are discussed.