Answers are grounded in this paper and cite their sources.
Traditional mural image inpainting methods mostly rely on visual information from known image areas, suffering from weak semantic understanding, missing damaged content, and low consistency between restoration outputs and textual semantic descriptions. To tackle these challenges, this paper designs a Masked Autoencoder equipped with noise-adaptive gated attention (MAE GA) for multimodal cultural heritage image restoration. Joint weighted optimization is performed over pixel reconstruction, multi-granularity semantic alignment, perceptual quality, and spatial continuity. Evaluated on the Dunhuang mural subset from the CulTi multimodal dataset, our model obtains PSNR = 30.45 dB, SSIM = 0.938, and Recall@1 = 0.658 for the text-image retrieval task. Comprehensive ablation experiments demonstrate that the proposed semantic and spatial constraints consistently improve semantic recall, perceptual quality, and edge continuity, while introducing an interpretable trade-off in pixel-level PSNR/SSIM: the configuration without alignment achieves higher PSNR/SSIM, but exhibits substantially lower Recall@1, poorer FID, and weaker edge continuity. Five-fold cross-validation, repeated random-seed experiments, and robustness experiments under multiple degradation patterns further demonstrate the model's stable generalization capability. Subjective evaluations by experts confirm the high texture fidelity and semantic plausibility of the restored outputs. This study further examines performance degradation when textual descriptions are inaccurate, incomplete, ambiguous, or unavailable, and discusses the corresponding limitations. The proposed algorithm provides a feasible technical reference for multimodal image restoration of cultural heritage artifacts.
No takes yet. Share an insight, caveat, or question.
Zhang et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: