Image Deraining Using Transformer Network with Sparse Non-Local Self-Attention

In recent years, Transformer architectures have excelled at modeling non-local information. This makes them suitable for image deraining. However, existing methods use dense self-attention. They compute all similarities between query and key tokens. This is inefficient. In practice, this approach can lead to the neglect of the most relevant information and result in a blurring effect of irrelevant representations during the feature aggregation process. To address this issue, this paper proposes an image deraining Transformer based on sparse non-local self-attention. The core of the network consists of multiple non-local feature extraction modules, primarily comprising a sparse self-attention network and a sparse feedforward network along the channel dimension. Specifically, we implement sparse attention by selecting the most useful similarities based on Top-k approximations. Furthermore, we have developed a sparse feedforward network to achieve more accurate representations for high-quality preservation results. Extensive experiments on benchmark datasets have demonstrated the effectiveness of our proposed method.