INF‐SLiM: Large‐Scale Implicit Neural Fields for Semantic LiDAR Mapping of Embodied AI Agents

ABSTRACT Large‐scale semantic mapping is crucial for outdoor autonomous agents to perform high‐level tasks such as planning and navigation. In this paper, we propose a novel method for large‐scale 3D semantic reconstruction through implicit representations from posed LiDAR measurements alone. We first construct a neural fields via the octree and latent features, then decode implicit features into signed distance value and semantic information through shallow Multilayer Perceptrons (MLPs). We leverage radial window self‐attention networks to predict the semantic labels of point clouds. We then jointly optimize the feature embedding and MLP parameters with a self‐supervision paradigm for point‐cloud geometry and a pseudo‐supervision paradigm for semantic and panoptic labels. Subsequently, geometric structures and semantic categories for novel points in the unseen area are regressed, and the marching cubes method is exploited to subdivide and visualize scenes in the inferring stage, the labeled mesh is produced correspondingly. Experiments on two real‐world datasets, SemanticKITTI and SemanticPOSS, demonstrate the superior segmentation efficiency and mapping effectiveness of our framework compared to existing 3D semantic LiDAR mapping methods.