GNeRF: Accelerating Neural Radiance Fields Inference via Adaptive Sample Gating

This is the authors' abstract. We don't add key points for this paper.

NeRF is an emerging algorithm in computer graphics that has achieved state-of-the-art results in areas such as image rendering and 3D reconstruction. However, to compute the RGB of pixels in a view, NeRF executes MLP calculations on a huge number of sample points, resulting in significant computational complexity. To address this issue, we propose a simple and hardware-friendly NeRF algorithm (dubbed GNeRF) in this paper. GNeRF is designed based on the concept of "gating-by-decomposing". Specifically, It decomposes the original large MLP into two smaller branches. For each ray, GNeRF utilizes one branch to predict the important samples based on the direction information adaptively. The RGB calculations are then solely performed on these important samples using the other branch. Experimental results show that GNeRF can achieve comparable PSNR with only 3% FLOPS of the original NeRF. To showcase the hardware efficiency of GNeRF, we also design an FPGA-based NeRF accelerator on Xilinx ZCU102 MPSoC. Evaluation reveals that GNeRF can significantly enhance inference performance with minimal modifications to the existing MLP engine.