User-Interactive Single-View 3D Reconstruction of Indoor Environments and Furniture based on SAM3D

This paper presents an end-to-end interactive interior redesign system that automatically segments, completes, and reconstructs furniture from a single indoor photograph for interactive 3D placement. Existing commercial interior applications are fundamentally limited to their own 3D furniture catalogs, preventing users from incorporating their existing furniture. To overcome this limitation, the proposed system consists of a SAM2-based instance segmentation pipeline, a two-stage empty room reconstruction pipeline using LaMa and Stable Diffusion ControlNet (Canny edge), a SAM3D-based furniture 3D reconstruction pipeline, and an interactive placement system built on Three.js. Experiments demonstrate that the vertex color rendering method produces no color bleeding artifacts and maintains stable processing time regardless of furniture geometric complexity, proving far superior for real-time systems compared to UV texture baking. Furthermore, we conducted an exploratory experiment applying SAM3D to full-room 3D reconstruction. However, due to the convex single-object shape prior, mesh generation failures on walls and floors as well as color inconsistency issues occurred in the concave indoor space. While trimesh post-processing and plane snap partially mitigated these problems, a complete solution remained elusive. As an alternative, we adopted a Three.js-based procedural 3D room construction approach that extracts color and texture directly from the input room image; complete SAM3D-based 3D room reconstruction remains a direction for future work. Finally, a user study with 20 participants was conducted to quantitatively validate the practical utility of the proposed system.