Improving LLM Performance for Construction Site Risk Factor Identification Using RAG and Fine-tuning

With the recent revision of the Occupational Safety and Health Act and the enforcement of the Serious Accidents Punishment Act, safety regulations have become more stringent, leading to a shortage of safety management personnel and increased documentation burdens on construction sites. This issue is especially severe at small- and medium-sized sites, where ensuring effective safety management is more challenging. In response, LLM-based task automation technologies have gained attention. However, general-purpose LLMs often lack the domain-specific knowledge needed for practical application in construction safety. To address this, the study develops an LLM specialized in identifying hazardous and harmful factors on construction sites by applying Retrieval-Augmented Generation (RAG) and fine-tuning methods. Experimental validation was conducted using OpenAI’s GPT model under three configurations: fine-tuning only, RAG only, and a combined fine-tuning + RAG approach. Performance was assessed through quantitative metrics (BLEU, ROUGE, BERT Score) and qualitative evaluations via user surveys. The combined approach showed the best performance, consistently outperforming general-purpose LLMs in all quantitative metrics and receiving the highest rating in the efficiency category of the qualitative evaluation. These results demonstrate that a domain-specialized LLM can significantly enhance hazard identification capabilities, supporting more effective safety management practices in construction. This study offers both academic and practical contributions by demonstrating a method for improving LLM performance in specialized domains and providing a technical foundation for automated tools in construction safety management.