FlexNeRFer: A Multi-Dataflow, Adaptive Sparsity-Aware Accelerator for On-Device NeRF Rendering

This is the authors' abstract. We don't add key points for this paper.

Neural Radiance Fields (NeRF), an AI-driven approach for 3D view reconstruction, has demonstrated impressive performance, sparking active research across fields.As a result, a range of advanced NeRF models has emerged, leading on-device applications to increasingly adopt NeRF for highly realistic scene reconstructions.With the advent of diverse NeRF models, NeRF-based applications leverage a variety of NeRF frameworks, creating the need for hardware capable of efficiently supporting these models.However, GPUs fail to meet the performance, power, and area (PPA) cost demanded by these on-device applications, or are specialized for specific NeRF algorithms, resulting in lower efficiency when applied to other NeRF models.To address this limitation, in this work, we introduce FlexNeRFer, an energy-efficient versatile NeRF accelerator.The key components enabling the enhancement of FlexNeRFer include: i) a flexible network-on-chip (NoC) supporting multi-dataflow and sparsity on precision-scalable MAC array, and ii) efficient data storage using an optimal sparsity format based on the sparsity ratio and precision modes.To evaluate the effectiveness of FlexNeRFer, we performed a layout implementation using 28nm CMOS technology.Our evaluation shows that FlexNeRFer achieves 8.2243.3speedup and 24.1520.3improvement in energy efficiency over a GPU (i.e., NVIDIA RTX 2080 Ti), while demonstrating 4.286.9speedup and 2.347.5improvement in energy efficiency compared to a state-of-the-art NeRF accelerator (i.e., NeuRex).