Geometry enhanced 3D Gaussian Splatting for high quality deferred rendering

Reconstructing and rendering the real world with high visual fidelity is crucial for VR applications. 3D Gaussian Splatting (GS) method offers high-quality novel views but struggles with realistic shadows under complex lighting, including both directional lighting and point lighting. We proposed a novel geometry enhanced 3D GS method, in which 3D Gaussians are learned efficiently with additional attributes of both normal and depth. The proposed representation enhances high-fidelity novel view rendering and integrates seamlessly into commercial real-time engines. We developed a deferred rendering pipeline for rasterization, enabling real-time complex illumination effects with high-precision depth and normal attributes learned by our 3D GS method. Our pipeline surpasses previous 3D GS renderers in accurate illumination, shadows, and directional lighting. Applied to VR applications like live broadcasting, it generates immersive virtual environments in real-time on consumer devices. Experimental results show our method delivers superior realistic rendering at real-time speeds, benefiting numerous VR applications.