Refracting Once is Enough: Neural Radiance Fields for Novel-View Synthesis of Real Refractive Objects

Neural Radiance Fields (NeRF) have shown promise in novel view synthesis, but it still face challenges when applied to refractive objects. The presence of refraction disrupts multiview consistency, often resulting in renderings that are either blurred or distorted. Recent methods alleviate this challenge by introducing external supervision, such as mask images and Index of Refraction. However,acquiring such information is often impractical,limiting the application of NeRF-like models to complex scenes with refracting elementsand yielding unsatisfactory results. To address these limitations, we introduce RoseNeRF (Refracting once is enough for NeRF), a novel method that simplifies the complex interaction of rays within objects to a single refraction event. We design the refraction network that efficiently maps a ray in the 4D light field to its refracted counterpart, better modeling curved ray paths. Furthermore, we introduce a regularization strategy to ensure the reversibility of optical paths, which is anchored in physical world theorems. To help it easier for the network to learn the highly view-dependent appearance of refractive objects, we also propose novel density decoding strategies. Our method is designed for seamless integration into most NeRF-like frameworks and has demonstrated state-of-the-art performance without any additional information on both the Eikonal Fields' dataset and Shiny dataset.