Abstract Hand‐drawn model sheets are widely used in character design to define 3D shape and appearance through sparse multi‐view drawings. Reconstructing 3D characters from such sparse inputs has traditionally been challenging due to insufficient visual information. Recently, 3D generative models have enabled automatic reconstruction of plausible 3D characters by learning from large‐scale training data, but achieving reconstructions that accurately match the input model sheets remains limited. In this paper, we present a framework that leverages the power of a 3D generative model for initial reconstruction and enhances the output to faithfully reproduce input model sheets. For such faithful reconstruction, we must address two fundamental challenges: (1) the hand‐drawn nature inherently introduces multi‐view inconsistencies where the generated 3D geometry cannot perfectly align with all views, and (2) view‐dependent line elements along geometry boundaries interfere with accurate texture reconstruction. To address these challenges, we optimize the geometry to minimize multi‐view inconsistencies and introduce a deformable per‐pixel camera ray representation that resolves residual discrepancies in cross‐view correspondences. We also decompose drawings into three distinct layers of view‐dependent lines, view‐independent colors, and fine‐detail decals to separately handle view‐dependent and view‐independent components for consistent cross‐view reconstruction. Comprehensive experiments demonstrate that our method outperforms possible alternatives regardless of the choice of 3D generative model, while successfully preserving both artistic intent and visual fidelity of input model sheets.
Yoon et al. (Fri,) studied this question.