An Improved Multi‐Task Pointwise Network for Segmentation of Building Roofs in Airborne Laser Scanning Point Clouds

Abstract Roof plane segmentation is an essential step in the process of 3D building reconstruction from airborne laser scanning (ALS) point clouds. The existing approaches either rely on human intervention to select the appropriate input parameters for different data‐sets or they are not automatic and efficient. To tackle these issues, an improved multi‐task pointwise network is proposed to simultaneously segment instances (that is, individual roof planes) and semantics (that is, groups of roof planes with similar geometric shapes) in point clouds. PointNet++ is used as a backbone network to extract robust features in the first step. The features from semantics branch are then added to the instance branch to facilitate the learning of instance embeddings. After that, a feature fusion module is added to the semantics branch to acquire more discriminative features from the backbone network. To increase the accuracy of semantic predictions, fused semantic features of the points belonging to the same instance are aggregated together. Finally, a mean‐shift clustering algorithm is employed on instance embeddings to produce the instance predictions. Furthermore, a new roof data‐set (called RoofNTNU) is established by taking ALS point clouds as training data for automatic and more general segmentation. Experiments on the new roof data‐set show that the method achieves promising segmentation results: the mean precision (mPrec) of 96.2% for the instance segmentation task and mean accuracy (mAcc) of 94.4% for the semantic segmentation task.