Semantic and Geometric Datasets Creation for Monument Analysis: A Framework for Expert-Guided Annotation and AI Models

Analyzing multimodal cultural heritage data, such as 2D images, 3D point clouds, and textual documentation, is essential for monument preservation but presents significant challenges due to data heterogeneity and the scarcity of annotated datasets. To address these limitations, we present a framework for Expert-Guided Annotation and Artificial Intelligence (AI) Models in Monument Analysis. Built upon the Notre-Dame de Paris digital ecosystem, our system enables domain experts to create high-quality, AI-ready datasets. Our framework introduces: (1) an enhanced, semi-automated annotation workflow capable of propagating expert annotations across modalities, increasing dataset volume by a factor of ten; (2) integrated 2D–3D visualization tools for intuitive annotation inspection and refinement; and (3) a standardized export system that produces datasets with rich semantic, geometric, and visual metadata in formats compatible with deep learning frameworks. We validate our system by generating annotations for degradation phenomena, architectural components, and decorative elements of Notre-Dame de Paris, and demonstrate their utility through integration with the Segment Anything Model (SAM) for semantic segmentation. Our approach bridges the gap between heritage expertise and computer vision, establishing a robust infrastructure for scalable, reproducible analysis and documentation.