Make it till you fake it: Construction-centric computational framework for simultaneous image synthetization and multimodal labeling

This paper introduces BlendCon, a fully automated framework capable of simultaneously synthesizing and labeling construction imagery data. This framework simulates a construction site by orchestrating 3D mobile objects against a 3D background and produces multimodal labels for target entities. The effectiveness of the synthetic data in training object detection models was thoroughly validated. For the construction worker detection task, a YOLOv7 model trained with synthetic data nearly matched the performance of a model trained with real data: it achieved 71% [email protected]–0.95 compared to 75% for the real data-trained model. Moreover, the model trained with synthetic data surpassed its real data counterpart in scenarios requiring stricter IoU thresholds, particularly above 85%. Acquiring a sufficient quantity and diverse range of imagery data has been a primary challenge in construction studies that focus on automation and digitization through deep neural networks. BlendCon can significantly contribute to addressing this data scarcity challenge.