NeRFUS: neural radiance fields with uncertainty and semantics

In recent years, significant progress has been made in the differentiable representation of three-dimensional scenes using neural radiance fields (NeRFs). These models can effectively synthesize novel views containing color information or semantic masks; however, they do not provide confidence estimates associated with these predictions. This limitation poses serious challenges for deployment in safety-critical domains, where uncertainty quantification is essential to avoid catastrophic failures. In this work, we propose NeRFUS, a novel NeRF-based model that enables each point in three-dimensional space to store uncertainty estimates alongside semantic information. For the first time, we integrate both aleatoric and epistemic uncertainty quantification for semantic label prediction into a NeRF framework. To achieve this, we adapt the evidential learning approach, which also enables the detection of objects whose classes are not represented in the training dataset. Comprehensive experiments across various scenes demonstrate the effectiveness of the proposed method and confirm the reliability of its predictions. The code is publicly available at https://github.com/yuddim/NeRFUS .