Self-similar feature aggregation for light field spatial super-resolution

With the advancement of deep neural networks, transformer-based methods have become the mainstream approach for light field (LF) spatial super-resolution (SR) tasks. However, the existing transformer-based LF spatial SR methods rely too much on global features in the extraction of non-local information, and the attention calculation is interfered by noise. In order to solve these problems, we propose an LF spatial SR network, SSFA-Net, based on self-similar feature aggregation. Specifically, we design a high-similarity feature aggregation transformer module. The module effectively aggregates highly similar feature information in sub-aperture, macro pixel, and epipolar-plane images through an adaptive selection strategy, which reduces the introduction of noise and mitigates the computational redundancy. Furthermore, in order to enhance the high-frequency details of LF images and maintain the consistency of all views, we propose a spatial-frequency fusion module (SFFM). The module restores the structural details in the spatial domain and enhances the high-frequency information in the frequency domain by utilizing their complementary advantages. Additionally, the channel branch of the SFFM enhances features containing key angular information, which helps to maintain the angular consistency across all views. Experimental results show that the proposed method effectively balances the accuracy and model complexity. It can reconstruct LF images with rich texture details, and is competitive in terms of peak signal-to-noise ratio, structural similarity, and learned perceptual image patch similarity. In addition, the proposed method effectively improves the performance of visual tasks, such as LF depth estimation and refocusing tasks.