Static-Scene Video Compression with Neural Radiance Fields

We study how to efficiently compress static-scene video, which captures a static scene from continuous viewpoints. The static-scene videos have emerged as a key ingredient for virtual/augmented reality. For static-scene videos, the existing video coding schemes such as H.265/HEVC may have limited compression efficiency due to the limited number of reference frames, which prevents the sufficient utilization of the temporal correlation. We build a static-scene video compression scheme using the recently developed technologies of Neural Radiance Fields (NeRF). The scheme is shown in Figure 1 . In our proposed scheme, the encoder derives and encodes the camera parameters, and compresses some selected keyframes; the decoder adopts an efficient NeRF algorithm to build an implicit scene model for reconstructing all the frames. To the compression of the camera parameters, we design a algorithm referring to the method of motion vector compression in video coding. It uses the reconstructed parameters of the previous frame plus the difference between the reconstructed parameters of the previous two frames as predictions and then quantify residuals to reduce bitrates.