Zero-shot Learning-based Polygon Mask Generation for Construction Objects

Zero-shot Learning-based Polygon Mask Generation for Construction Objects Taegeon Kim, Minkyu Koo, Jeongho Hyeon, Hongjo Kim Pages 81-88 (2024 Proceedings of the 41st ISARC, Lille, France, ISBN 978-0-6458322-1-1, ISSN 2413-5844) Abstract: For construction sites monitoring, the use of segmentation-based computer vision technology has been proposed. In such environments, the main technical challenge is the generation of data for training the segmentation model. The training data for a segmentation model involves polygon annotation of objects within an image, which is a time-consuming task. To address this issue, this study proposes a new approach that uses the YOLOv8 object detection model to predict bounding box labels and inputs these into a Segment Anything Model (SAM) to automatically generate polygon label data. The performance of the YOLOv8 model exceeded 80%, and the automatic generation of polygon labels through SAM resulted in an IoU range of 55-86%, producing high-quality mask label data. This approach significantly reduces the time, labor, and cost associated with the labeling process. Keywords: Polygon label generation, Instance segmentation, Zero-shot learning DOI: https://doi.org/10.22260/ISARC2024/0012 Download fulltext Download BibTex Download Endnote (RIS) TeX Import to Mendeley