Integrating Visual and Contextual Cues for Ancient Chinese Rubbing Image Restoration

Rubbing served as a crucial cultural carrier, transferring characters from steles onto paper using ink, thereby facilitating greater dissemination and longer-term preservation. However, prolonged corrosion, inclement weather, and other factors cause various rubbing deteriorations. Therefore, an automated rubbing image restoration method is highly desirable to reduce the complexity of manual restoration procedures. In particular, when we refer to restoring a rubbing image, we mean recreating the original text on the rubbing image to reflect its original appearance. Existing methods, such as image denoising, can address minor deterioration but cannot handle severe deterioration. Image inpainting methods attempt to handle large areas of deterioration but generally struggle to produce contextually consistent strokes due to a lack of control in the restoration process. Additionally, currently there is no reliable method for rubbing character recognition. In this paper, we propose a novel two-stage rubbing image restoration method that performs well on different levels of deterioration. In the first stage, we integrate visual and contextual information for text recognition in rubbing images, including those that are heavily deteriorated. The second stage includes a multi-prior conditional latent diffusion model for rubbing image restoration, utilizing the character identities recognized in the first stage and undeteriorated characters as priors to guide the restoration process. Extensive experiments have demonstrated the effectiveness of our method compared to existing approaches.