CAFformer: channel-based adaptive fusion with transformer for reference-based super-resolution

Reference-based Super-Resolution (RefSR) has garnered significant attention for its capacity to leverage external a priori information. RefSR involves the intricate process of transferring texture details from a Reference (Ref) image to a LowResolution (LR) image, relying on corresponding pixel or patch relationships. Despite the proliferation of Convolutional Neural Network (CNN) or Transformer-based models aimed at enhancing RefSR performance, a substantial number of approaches neglect the distinct roles played by LR and Ref images within the reconstruction process. Specifically, LR images inherently contain fundamental structural information corresponding to High-Resolution (HR) images, while Ref images encapsulate potential high-frequency details. In this paper, we introduce a novel module for channel-based adaptive fusion, specifically designed to integrate features extracted from LR and Ref images. The proposed module adeptly combines LR and Ref image features along the channel dimension, enabling the efficient utilization of Transformer long-range modeling capabilities across the width and height dimensions of the feature map. The incorporation of this innovative module results in superior performance compared to state-of-the-art Transformer-based methods, concurrently demonstrating an improvement in inference speed. Rigorous experimental results validate the efficacy of our approach, Channelbased Adaptive Fusion with Transformer for Reference-based Super-Resolution (CAFformer), outperforming existing methods in both quantitative and qualitative evaluations. This contribution holds promise for advancing the field of superresolution reconstruction by comprehensively addressing the inherent distinctions between LR and Ref images.