Sgrgan: sketch-guided restoration for traditional Chinese landscape paintings

Abstract Image restoration is a prominent field of research in computer vision. Restoring broken paintings, especially ancient Chinese artworks, is a significant challenge for current restoration models. The difficulty lies in realistically reinstating the intricate and delicate textures inherent in the original pieces. This process requires preserving the unique style and artistic characteristics of the ancient Chinese paintings. To enhance the effectiveness of restoring and preserving traditional Chinese paintings, this paper presents a framework called Sketch-Guided Restoration Generative Adversarial Network, termd SGRGAN. The framework employs sketch images as structural priors, providing essential information for the restoration process. Additionally, a novel Focal block is proposed to enhance the fusion and interaction of textural and structural elements. It is noteworthy that a BiSCCFormer block, incorporating a Bi-level routing attention mechanism, is devised to comprehensively grasp the structural and semantic details of the image, including its contours and layout. Extensive experiments and ablation studies on MaskCLP and Mural datasets demonstrate the superiority of the proposed method over previous state-of-the-art methods. Specifically, the model demonstrates outstanding visual fidelity, particularly in the restoration of landscape paintings. This further underscores its efficacy and universality in the realm of cultural heritage preservation and restoration.