SGN: Semantic-Guided Net for 3D Semantic Segmentation in Cluttered Scenes

3D semantic segmentation is a fundamental task in robotic manipulation and computer vision. However, due to the limitation of heuristic point cloud upsampling methods, the performance of 3D semantic segmentation under cluttered conditions is far from satisfactory, which severely limits the performance of robotic grasping. Specifically, for each query point, previous point-based upsampling methods only choose k nearest neighbors in 3D space as the point set for interpolation, which may cause ambiguity in predicting the category of the query point. To tackle this issue, this paper proposes a novel point cloud semantic segmentation network called Semantic-Guided Net (SGN). The key to SGN is Semantic-Guided Upsampler (SGUS). SGUS make use of both 3D position and semantic information to find the neighbors for interpolation, which reduces the number of irrelevant points from different categories. Furthermore, this paper presents a 3D semantic segmentation dataset called Semantic Cluttered Objects (SCO) to test the proposed module under robotic vision grasping circumstances. Extensive experiments on S3DIS and SCO show the effectiveness of SGN.