On the Development and Application of a Structured Dataset for Data-Driven Risk Assessment in Industrial Functional Safety

This paper introduces and makes available the first structured dataset for data-driven risk assessment in industrial machinery functional safety. The dataset comprises 7800 hazard scenarios generated using a rigorous, ISO 12100 and ISO 138491 grounded methodology, including plausibility assessment. The dataset spans ten hazard types, incorporating contextual parameters and corresponding required performance level (PLr) for each hazard scenario. The utility of the dataset and its application as a benchmark is demonstrated by comparing LLM-only and RAG-enhanced hazard classification as one example application. Experiments show that that RAG significantly improves LLM accuracy in classifying hazard scenarios and predicting PLr by providing access to critical contextual information (e.g. guidance from ISO 13849-1 for determination of PLr), resulting in a 36.4% average accuracy increase (72.8% accuracy) compared to the LLM-only baseline (36.4% accuracy). Furthermore, RAG reduced misclassification rates across all hazard types, with the highest misclassification dropping from over 70% in the LLM-only case to a maximum of 29% with RAG indicating significantly enhanced classification confidence confirmed by statistical tests. These findings demonstrate the dataset’s ability to address the gap in structured data for evaluating LLMs in functional safety tasks, potentially leading to more reliable and efficient automated risk assessments. This work provides a valuable, open-source resource (GitHub repositry) to facilitate collaborative research and adoption of AI in routine functional safety workflows.