What type of study is this?

September 5, 2025Open Access

A Domain Knowledge-Enhanced Large Vision-Language Model for Construction Site Safety Monitoring

CCChun Man ChanHong Kong Green Building Council XGXiaowen GuoHong Kong University of Science and Technology PWPeter Kok-Yiu WongHong Kong University of Science and Technology

Key Points

Accuracy in identifying safety violations reached around 90%, validating the model's effectiveness in real-world applications.
The model employs curriculum learning to integrate construction-specific knowledge, improving hazard detection capabilities.
A semi-automatic image-text labeling pipeline streamlines data annotation, enhancing the efficiency of safety monitoring processes.
The deployed system provides interactive visual question answering, facilitating immediate compliance inspection and feedback.

Abstract

To address the industry-wide and policy-driven requirements toward construction site safety monitoring, this paper develops a virtual assistant agent based on a large vision-language model (VLM), integrated into on-site surveillance camera system for real-time identification and alerting of unsafe worker behaviors. First, we designed a semi-automatic image-text labeling pipeline, employing in-context learning to enhance data annotation efficiency. Then, we established a two-stage curriculum learning paradigm to deeply embed construction domain knowledge into the VLM, which is eventually embedded into a real-time video analytical engine for safety compliance inspection and interactive visual question answering. The system has been deployed on a real construction site, with around 90% accuracy in identifying violations of work-at-height safety regulations.

KI fragen

Bookmark

View Full Paper