Generating Building Information Models via computer-use agents

Building Information Modeling (BIM) authoring in the Architecture, Engineering, and Construction (AEC) sector entails navigating complex professional software, demanding significant manual effort and domain expertise. Current automation approaches rely on rigid Application Programming Interfaces (APIs), which detach agents from human-centric visual workflows and limit generalizability across software platforms. This paper presents BIMgent, a multimodal LLM-based agentic framework designed to perform autonomous BIM authoring directly via Graphical User Interface (GUI) interactions. To address high visual noise and long-horizon dependencies, BIMgent incorporates documentation-guided hierarchical planning and dynamic GUI grounding. The framework is evaluated on a benchmark derived from real-world designs of residential buildings. BIMgent achieves a 77.8% end-to-end success rate on long-horizon BIM authoring tasks, significantly outperforming leading generalist computer-use models, including Claude-Sonnet-4.5 (2.2%) and GPT-5.4 (11.1%). By bridging abstract design inputs and concrete modeling actions without API constraints, BIMgent offers a viable path towards autonomous architectural modeling assistants. • A LLM-based multi-agent framework that builds 3D BIM models via GUI, not APIs. • Dynamic GUI grounding, RAG-based planning, and self-reflective execution for GUI-based authoring. • A benchmark of 45 BIM authoring tasks with over 4800 GUI actions. • 77.8% success rate, significantly outperforming leading generalist computer-use models. • A new paradigm for domain-specific LLM agents leveraging human-centric engineering software.