ControlScene: Controllable Text‐to‐3D Scene Generation via Structured Layout Priors

Text‐to‐3D scene generation has emerged as a promising approach for enabling natural and intuitive 3D content creation through language. However, existing methods often struggle with generating scenes that are semantically faithful, spatially coherent and controllable, primarily due to the lack of grounded spatial reasoning and fine‐grained structural supervision. In this paper, we present ControlScene: controllable text‐to‐3D scene generation via structured layout priors, a novel framework that incorporates layout‐level structure into the generation process, allowing LLM to jointly reason about semantics and spatial composition. To support this framework, we introduce LayoutVerse‐20K, a large‐scale benchmark dataset comprising 20,000 manually annotated samples, each including textual prompts, layout metadata, graph‐structured layouts, scene graphs and corresponding 3D scenes. For fine‐grained evaluation, we propose two task‐specific metrics: category and layout plausibility, which assess semantic and spatial alignment by comparing generated outputs against common sense priors derived from top‐k reference samples. Additionally, we develop an interactive user interface for real‐time layout customisation, enhancing controllability and practical usability. Extensive experiments show that ControlScene significantly outperforms baseline methods in spatial realism, semantic consistency and user controllability, providing a robust foundation for future research in grounded language‐driven 3D scene generation. Project site: https://winkquan.github.io/ControlScene‐Page .