Sketch2Factory: Roadmap for AI-Driven Sketch to 3D-Model Pipeline for Factory Planning

Factory planning is shifting from static 2D layouts to living, data-rich digital twins that must reflect machinery footprints, safety constraints, and dynamic workflows. Yet most automated 2D-to-3D methods are tuned to residential settings, constrained by semantically thin datasets and pipelines that generalize poorly to industrial environments. This paper proposes a roadmap for an AI-driven sketch-to-3D pipeline that transforms hand-drawn factory layouts into high-fidelity, parameterized 3D models ready for digital-twin use. The roadmap integrates three pillars— data-centric enhancements, domain-knowledge infusion via ontologies and rules, and advanced geometric/spatial reasoning—to close the automation gap for factory planning. Our analysis draws on an end-to-end prototype (CNN vectorization — LLM semantic parsing — Blender reconstruction), which performs robustly on residential plans but exposes failure modes on factory layouts, underscoring the need for domain-specific data and semantics. Implementing the roadmap promises faster planning cycles, improved safety/compliance, and real-time digital-twin updates for Industry 4.0.