DS-GS: Depth guide structured 3D Gaussian for real-time rendering

Neural rendering methods represent a crucial and challenging task in computer vision. Recent advancements in 3D Gaussian Splatting have set new benchmarks in rendering quality and speed by combining the strengths of geometric primitives and volumetric representations. In contrast to the point-based approach of 3D Gaussian Splatting, Scaffold-GS obtains anchors to achieve significant memory reduction. In this paper, we use Scaffold-GS as the baseline to achieve high quality rendering with minimal anchors. We propose a depth guide structured 3D Gaussian method for real-time rendering, called DS-GS. Our method employs depth priors to guide two types of anchor growth, ensuring that anchors develop in the correct positions during training. And we implement a differentiable depth rasterizer to enhance the similarity between rendered depth and estimated depth. Additionally, we use multi-scale training to improve rendering quality. We demonstrate the advantages and potential of our approach across a variety of scenarios, effectively reducing anchor redundancy and improving rendering quality. For intance, in the Mip-NeRF360 dataset, we increased the PSNR by 0.6 and reduced the storage size by 30%. Furthermore, our approach shows robustness in experiments with real-world scenes.