Zero-shot building footprint extraction and regularization based on Segment Anything model with Mesh Model

Abstract. With the advancement of urbanization, building footprint data plays an important role in urban planning, 3D Real Scene and smart cities. Traditional manual contouring methods are time-consuming and laborious, while deep learning-based building extraction methods often require a large amount of labeled data and have limited generalization ability. In this paper, a zero-shot framework based on Segment Anything Model (SAM) is proposed for extracting and regularing building footprints from 3D mesh data. The method mainly consists of three steps: 1) Coarse Prompt Generation, irrelevant element’s masks such as ground and vegetation are eliminated by semi-global filtering and traditional classification method, and rough building mask is obtained as a boundary box prompt. 2) Fine mask generation: Using SAM's mask prompt capability, combined with logits map and grid elevation information with adaptive threshold to generate the fine mask prompt. Combine it with the updated bounding box to form hybrid prompt, and input SAM to generate a refined building mask. 3) Footprint regularization: Kinetic Partition, Markov random field, and Region Growth Algorithm are used to extract regularized building contours. Structural line segments from LSD guide the Kinetic Partitioning of the building. Markov random field matches building labels, while a region growth-based boundary reassignment refines the contours. The final regularized contour integrates the partitioned building zones. Our method achieved 78.31% AP50 on the Vaihingen dataset and obtained regular footprints that closely align with the true building contours on real Mesh data.