Semantic Segmentation of TLS Colored Point Cloud via Cube Map Projection with RGB Image-Assisted Learning

In recent years, a wide variety of three-dimensional (3D) scanning devices have been developed, allowing users to select appropriate sensing systems depending on their objectives and operational constraints. Among these devices, terrestrial laser scanners (TLS) offer superior scanning quality and accuracy and continue to play an important role in 3D sensing. Within TLS point cloud processing, object recognition and semantic segmentation are regarded as particularly important tasks. Numerous methods have been proposed for this purpose, and in recent years, approaches based on deep learning have attracted considerable attention. However, point cloud data are inherently irregular, unstructured, and unordered, which can make it challenging for existing methods to fully exploit their spatial information and consistently achieve satisfactory performance. In this study, we focus on TLS point clouds with RGB information and propose a semantic segmentation framework based on cube map projection. In the proposed framework, the 3D point cloud is first projected into a cube map representation to generate images corresponding to each face of the cube. Semantic segmentation is then applied to the generated images using mature two-dimensional (2D) convolutional neural networks (CNNs). The resulting semantic labels are subsequently reprojected onto the original point cloud, thereby achieving semantic segmentation of the 3D point cloud. The first advantage of the proposed framework is that it enables the application of well-established 2D CNN-based segmentation methods to the inherently challenging task of 3D point cloud segmentation. The second advantage is that RGB images, which are relatively easy to acquire and annotate, can be used as training data instead of 3D point clouds. The effectiveness of the proposed framework is validated through a series of experiments and evaluations conducted on TLS point clouds acquired in forest environments.