A Study of Attention Simplification for Efficient Image Super-Resolution

Lightweight Swift Attention Network for Super-Resolution (LSAN-SR) is a lightweight super-resolution network that keeps the spirit of the Swift Parameter-free Attention Network while eliminating most of its computational burden. The main contribution is the LSAB block, which substitutes the original three-convolution attention unit with a single path consisting of point-wise expansion, large-kernel depth-wise convolution, and point-wise projection. Group Normalization and a squeeze-and-excite gate provide stable activation scaling, and a small residual-scaling factor prevents gradient explosions during deep stacking. Only the final reconstruction stages retain wide channel expansion, whereas earlier layers remain slim, reducing parameters and memory without compromising representational power. Training is further stabilised by cosine learning-rate annealing, gradient accumulation, and exponential-moving-average weights. Comprehensive experiments on public benchmarks confirm that LSAN-SR retains the quality–speed balance of its predecessor while running comfortably within mobile and edge-device constraints, demonstrating that careful normalization, selective channel widening, and large-kernel depth-wise filtering can reproduce parameter-free attention benefits in a deployment-oriented model.