Inferring representations of 3D scenes from 2D observations is a fundamental of computer graphics, computer vision, and artificial intelligence. 3D-structured neural scene representations are a promising approach to3D scene understanding. In this work, we propose a novel neural scene, Light Field Networks or LFNs, which represent both geometry and of the underlying 3D scene in a 360-degree, four-dimensional light parameterized via a neural implicit representation. Rendering a ray from LFN requires only a single network evaluation, as opposed to hundreds of per ray for ray-marching or volumetric based renderers in3D-structured neural scene representations. In the setting of simple scenes, we meta-learning to learn a prior over LFNs that enables multi-view light field reconstruction from as little as a single image. This results in dramatic reductions in time and memory complexity, enables real-time rendering. The cost of storing a 360-degree light field an LFN is two orders of magnitude lower than conventional methods such as Lumigraph. Utilizing the analytical differentiability of neural implicit and a novel parameterization of light space, we further the extraction of sparse depth maps from LFNs.
No takes yet. Share an insight, caveat, or question.
Sitzmann et al. (2021) studied this question.