Dataset-level reliability and quality control for metallurgical data structured with large language models

Data-driven alloy design depends on the availability of reliable, structured experimental information. Although the scientific literature contains large volumes of such data, heterogeneous reporting practices hinder their systematic reuse at scale. General-purpose large language models (LLMs) offer a flexible approach for structuring metallurgical text without domain-specific fine-tuning, but they introduce errors such as omissions, misassignments, and fabrications that are not fully characterised by conventional extraction metrics. This study examines error behaviour in LLM-assisted extraction of metallurgical information using a multi-stage workflow comprising text classification, structured extraction, and automated review. Controlled experiments are used to analyse how prompt design, task formulation, and reporting structure influence error behaviour within each stage of the workflow. The workflow is subsequently evaluated at large scale on a corpus of 23,861 full-text publications on high-chromium steels, combining automated reviewer signals with statistically grounded manual sampling to assess residual errors. The results identify dominant, task-dependent error modes and show that similar extraction performance can correspond to substantially different uncertainty profiles across information types. These findings clarify how error behaviour depends on text structure and task formulation and provide a basis for the risk-aware reuse of literature-derived metallurgical data.