KPGS: Towards Real-World Complex Dynamic Scene Rendering With Keyframe-Driven Predictable Gaussian Splatting

Rendering complex dynamic scenes offers the advantage of observing and understanding the real world. However, existing Dynamic Scene Rendering (DSR) methods remain challenged by suboptimal reconstruction fidelity. These limitations stem from relying on a single, unified deformation model, which struggles to capture complex motions involving multiple sub-motions and abrupt geometric transitions. While temporal decomposition methods could alleviate such shortcomings, they introduce the additional challenge of ignoring motion correlations and increasing storage requirements. To address these issues, we introduce Keyframe-driven Predictable Gaussian Splatting (KPGS)-an efficient framework for high-fidelity complex dynamic scene rendering. Firstly, we present a patch- wise HSV clustering for extracting keyframes. Secondly, a prediction network based on the Transformer is utilized to calculate the deformable Gaussians at discrete keyframe times via voxelization. Thirdly, we propose an inter-frame deformation network and a mutual supervision between adjacent segments to maintain the temporal continuity. Extensive experiments on our newly built dataset (MotionGS), as well as public benchmarks HyperNeRF and Neu3D, demonstrate that KPGS could achieve a higher average view synthesis performance than SOTA approaches, while maintaining a balance between storage cost and performance. More details of the demo and dataset are available at KPGS Supplementary.