IncrementalDreamer: Scene-level 3D Generation with Incremental Optimization

Abstract Efficient 3D generation has long been a focal point of research. However, scene-level 3D generation techniques continue to encounter significant challenges, particularly in sparse-view setups, where occlusion holes and visual artifacts remain persistent issues. To address these issues, we propose IncrementalDreamer, a novel framework for scene-level 3D generation based on 3D Gaussian Splatting. Our method introduces an iterative incremental optimization strategy that combines monocular depth priors with diffusion-based inpainting to explicitly identify and complete structurally missing regions in 2D space before lifting them into 3D. This approach effectively mitigates occlusion artifacts and geometric discontinuities caused by sparse-view initialization. We also design a staged training mechanism that progressively fuses new geometry into the existing scene representation, enhancing global consistency and reconstruction quality. Experimental results show that IncrementalDreamer achieves superior CLIP scores, outperforming existing methods in semantic alignment and geometric fidelity, while demonstrating strong efficiency and scalability. Our framework offers a practical, high-fidelity solution for the generation of controllable 3D scenes.