Comparative assessment of Neural Rendering methods for the 3D reconstruction of complex heritage sites in the inner areas of the Marche region - Italy

The attempt to accurately reproduce real-world objects and environments in three-dimensional digital form has always been a significant challenge in the field of Digital Cultural Heritage (DCH) Cotella ( 2023), Abergel et al. (2023).Traditional techniques such as Structured Light Scanning, Photogrammetry and Stereo Multi-View, while advanced for their time, were often limited in their ability to manage complex structures and capture the rich details of intricate surfaces Cianci and Colaceci (2023).Despite their benefits, these techniques are limited by the diverse materials, properties and large-scale intricate details of heritage sites, posing significant challenges in achieving comprehensive digital representations.Artificial Intelligence (AI) approaches to 3D reconstruction have brought a paradigm shift, enabling the creation of highly detailed and realistic models Khairina et al. (2023), Fontanella et al. (2020).Neural Radiance Fields (NeRF), in particular, represent a significant advancement in this field Mildenhall et al. (2021).NeRF synthesizes photorealistic images using a neural network to model the volumetric function of a scene, capturing complex light and shadow relationships.This technique surpasses traditional methods in handling occlusions and complex textures, generating continuous, detailed 3D representations even from limited image sets.NeRF is particularly beneficial for documenting fragile or inaccessible cultural heritage sites non-invasively Tavakoli et al. (2023).Despite these advances, a comprehensive and comparative evaluation of novel 3D reconstruction approaches in DCH domain remains a gap in the existing literature.Such an evaluation is essential to understand the relative strengths and weaknesses of each method and to determine their most appropriate application contexts.More specifically, the CH sector faces the challenges of defining low cost strategies for its documentation, while keeping details and precision as faithful as possible with reality.The recent literature uncovered new potential of Neural Rendering in this field, Croce et al. (2024b) though their performances in terms of reliability are partially uncovered.This paper aims to fill this gap by presenting a detailed comparative study of different Neural Rendering (NR) techniques: NeRF, Signed Distance Function (SDF) and 3D Gaussian Splatting Kerbl et al. (2023).We explore the state-of-the-art for each technology, with a strong focus on quality, training time and resource consumption, reflecting the latest research developments.The aim is to provide a comprehensive guide that compares the quantitative metrics and training characteristics of the networks that support these technologies.We developed a complete pipeline that starts from dataset acquisition, to camera pose estimation, to NR networks training, metrics evaluation and 3D mesh extraction.This comparative pipeline is intended to help practitioners make informed decisions about the most appropriate tools for their specific datasets and usage conditions.The outcome of this research presents a detailed analysis of the results, conditions of use and recommended applications for each method, distinguishing between amateur and professional applications, including those aimed at accurate surface reconstruction for both digital interaction and in-depth analytical studies of reconstructed morphologies.Furthermore, we have carefully collected datasets from heritage sites in the Marche region, located in the center of Italy.The motivation for this experiment lies in the broad territorial scope of the project itself.In fact, the contribution of the research group to the inner areas of the region is to promote the heritage, both from a landscape and architectural point