ST-gaussian: improved 3D content generation based on stable diffusion and gaussian splatting

To address the issues of high acquisition cost and low efficiency of 3D models, a single-view 3D model generation method based on a diffusion model and Gaussian Splatting is proposed. This method inputs a single image and supplements side views based on the diffusion model. On this basis, a 3D Gaussian Splatting (3DGS) reconstruction method based on a random view supervision strategy and a local point cloud adaptive completion mechanism is introduced to improve the reconstruction quality. The reconstructed model is rendered off-axis to generate dense viewpoint images, which are then encoded to achieve the generation of a 3D model from a single view. Experimental results show that the improved 3DGS outperforms the traditional 3DGS algorithm in terms of Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), and Learned Perceptual Image Similarity (LPIPS).