Enhanced 3D Monocular Reconstruction of Relief-Type Cultural Heritage via an Edge-Matching Module

This is the authors' abstract. We don't add key points for this paper.

Historical sites around the world often contain damaged reliefs, and efficient digital 3D reconstruction methods are urgently needed to enable the necessary repairs. Using monocular depth estimation, a single old photograph with pre-damage information can be utilized to restore the lost areas in the reliefs. However, reconstructing intricate features along low-curvature edges is challenging due to depth value compression. To address this problem, we propose a novel edge-matching module that constrains the depth estimation task in the edge regions and that is trained jointly with depth estimation in a multitask learning framework. This module enables the network to focus on the three-dimensional (3D) structural details of the damaged relief areas, thereby achieving more refined 3D reconstructions. We evaluate the effectiveness and zero-shot generalization performance of our method on four cultural heritage sites. The sites differ in damage severity, stylistic characteristics, and the age and quality of the old photographs used for inference. In addition, to increase the generalization ability of the proposed model, we introduce a patch-voting algorithm that enables the network to handle old photographs of varying scales, which slightly improves the relief reconstruction accuracy.