Digital protection and virtual restoration of cultural relics based on multimodal data

Abstract To address issues such as information loss and insufficient restoration accuracy in the process of cultural relic protection and restoration, this study proposes a digital protection and virtual restoration method for cultural relics that integrates multimodal data, aiming to enhance the intelligence and refinement level of cultural relic restoration. This method comprehensively utilizes multimodal data including pattern textures, structural lines, coloring, and material properties, and conducts virtual restoration based on the Gated Convolutional Generative Adversarial Network (GC-GAN) framework. The method reconstructs the structural lines of cultural relic patterns through adaptive curve fitting technology, and combines material attribute information to achieve accurate restoration of damaged areas, thereby improving visual consistency while maintaining historical authenticity. The GC-GAN consists of a coarse generator, a fine generator, and a discriminator, which effectively restore damaged images. Experiments show that GC-GAN outperforms traditional algorithms like Criminisi, especially in SSIM and PSNR metrics. The average improvements are 7.4 % and 30.5 %, respectively. In the overall validation of the virtual restoration method, this approach achieves better restoration effects and image quality than other methods. The average SSIM and PSNR improvements are 0.83 % and 6.19 %, respectively, with stable restoration results. Although the runtime is slightly longer than other methods, it still meets practical application needs. Tests with different missing rates demonstrate strong robustness at lower missing rates. The research results provide a feasible technical path and new ideas for the digital protection and virtual restoration of cultural relics, and possess significant practical application value and promotion potential.