In the construction industry, occupational health and safety plays a critical role in preventing occupational accidents and increasing productivity. In recent years, computer vision and artificial intelligence-based systems have made significant contributions to improving these processes through automatic detection and tracking of objects. The aim of this study was to fine-tune object detection models and integrate them with Large Language Models for (i). accurate detection of personal protective equipment (PPE) by specifically focusing on helmets and (ii). providing real-time recommendations based on the detections for supporting the use of helmets in construction sites. For achieving the first objective of the study, large YOLOv8/v11/v12 models were trained using a helmet dataset consisting of 16,867 images. The dataset was divided into two classes: “Head (No Helmet)” and “Helmet”. The model, once trained, was able to analyze an image from a construction site and detect and count the people with and without helmets. A tool with the aim of providing advice to workers in real time was developed to fulfil the second objective of the study. The developed tool provides the counts of the people based on video feeds or analyzing a series of images and provides recommendations on occupational safety (based on the detections from the video feed and images) through an OpenAI GPT-3.5-turbo Large Language Model and with a Streamlit-based GUI. The use of YOLO enables quick and accurate detections; in addition, the use of the OpenAI model API serves the exact same purpose. The combination of the YOLO model and OpenAI model API enables near-real-time responses to the user over the web. The paper elaborates on the fine tuning of the detection model with the helmet dataset and the development of the real-time advisory tool.
Işıkdağ et al. (2025) studied this question.