PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
March 13, 2026IEEE Transactions on Pattern Analysis and Machine Intelligence3 citations

StarIR: Convolutional Image Restoration With Spatial-Frequency Fusion

View Full Paper
YCYuning CuiSZSyed Waqas ZamirMYMing-Hsuan Yang

Key Points

  • The research aims to develop an efficient image restoration framework that combines the strengths of CNNs and Vision Transformers.
  • Proposed a dual-domain representation learning framework for spatial and frequency domains.
  • Implemented a high-dimensional feature fusion mechanism, called the Star operation, for enhanced capacity.
  • Incorporated a channel attention unit to improve global feature modeling.
  • Achieved state-of-the-art performance across 21 datasets in six single-degradation image restoration tasks.
  • Showed robustness on composite-degradation datasets and performed well in all-in-one settings.
  • Demonstrated effective applications in ultra-high-definition imaging, remote sensing, medical imaging, and underwater enhancement.

Abstract

Vision Transformer (ViT) has shown impressive performance in image restoration due to its ability to capture a large receptive field. However, its complexity grows quadratically with input resolution, limiting its applicability for high-resolution images. In contrast, Convolutional Neural Networks (CNNs) are computationally efficient but are constrained by their inherently local receptive fields, which limit their ability to capture long-range pixel relationships. To address these challenges, we propose StarIR, which possesses the efficiency of CNNs while also capturing a large receptive field, similar to Transformers. StarIR incorporates two key innovations: 1) a dual-domain representation learning framework, with one branch processing spatial details and the other focusing on mesoscale interactions in the frequency domain; and 2) a high-dimensional feature fusion mechanism, the Star operation, which fuses information from both domains through element-wise multiplication, thereby enhancing representational capacity without increasing network width and depth. Our Star operation is followed by a channel attention unit to facilitate global feature modeling and enhance channel-wise interactions. Building on our straightforward yet powerful design principles, StarIR achieves state-of-the-art performance across 21 datasets covering six single-degradation image restoration tasks. Furthermore, our model performs favorably against leading algorithms in two all-in-one settings and demonstrates robustness on two composite-degradation datasets. In addition, StarIR extends well to several domain-specific applications, including ultra-high-definition (UHD) imaging, remote sensing, medical imaging, and underwater image enhancement.

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

Cui et al. (2026) studied this question.

synapsesocial.com/papers/69b3abe702a1e69014ccd20ehttps://doi.org/10.1109/tpami.2026.3672465
Ask AI
Helpful
Bookmark
Share
View Full Paper