Object Detection and Instance Segmentation in Construction Sites

Identifying safety hazards and ensuring quality control in construction sites requires information from surveillance images, such as clear detection boxes, accurate edges of targets, etc. Although both object detection and instance segmentation models have yielded good results on large open datasets, there is still room for improvement in the construction field. Besides, previous methods based on traditional DNNs have encountered bottlenecks in improving performance, and the detection results are far from enough to be used in practice. We present the Cascade Mask R-CNN framework, which simultaneously handles object detection and instance segmentation tasks in construction scenarios using a single model. And the Swin Transformer model serves as the backbone of the Cascade model, which enhances feature maps extracted by the backbone and strengthens the model for construction sites. Additionally, we propose a post-processing method that refines the initial coarse instance masks generated by the Cascade model. Our experiments show that our model is highly effective at detecting objects and segmenting instances in construction sites, with a mean average precision (mAP) of 60.2% for object detection and 54% for instance segmentation.