3D Feature Engineering for Deep Semantic Segmentation of Indoor Point Clouds

Semantic segmentation of indoor point clouds is a common task for various applications such as Building Information Modeling (BIM), digital twins, and autonomous navigation. Deep learning on 3D point clouds made impressing progress in the last decade. However, segmentation methods based on deep models tend to overlook the classical geometric features used in photogrammetry, despite the fact that they could benefit model training and improve performance. We show that incorporating geometric features, such as linearity, omnivariance, and anisotropy, can improve the deep semantic segmentation of 3D point clouds on several indoor datasets. To improve the robustness of such models to scenarios where these features are not available, we introduce DropFeatures, a regularization strategy that randomly removes geometric features during training. We show that this not only prevents catastrophic performance drops when features are missing at inference time, but can also improve performance by acting as a data augmentation. For example, in S3DIS without features, our strategy helps maintain performance and avoids a drop of +35.58% mIoU, allowing models to perform 0.89% points better than the RGB baseline. We conduct experiments on three indoor point cloud semantic segmentation datasets and show that our results are consistent across datasets and model architectures.