MSA2T-Net: A Multiscale Attention Augmented Transformer Network for Hyperspectral Image Classification

Abstract Hyperspectral image classification is crucial in remote sensing but faces significant challenges, including long-range dependence in spatial-spectral information and the difficulty of effectively fusing spectral and spatial features. To address these issues, we propose a novel classification framework, Multiscale Attention Augmented Transformer Network (MSA²T-Net). The framework integrates three key modules: Dynamic Spatial Attention Unit (DSAU), Multi-Kernel Fusion Attention (MKFA), and Cross-Attention Swin Transformer (CASTB). These modules enhance feature representation, efficiently extract multi-scale features, and improve the integration of spatial and spectral information, which leads to improved classification consistency and robustness. Experimental results on four publicly available hyperspectral datasets (Pavia, Houston2013, PaviaU, and Salinas) demonstrate that MSA²T-Net outperforms state-of-the-art methods in overall accuracy, average accuracy, and Kappa coefficient. Ablation studies further confirm the effectiveness of each module. The proposed method offers a balanced solution for HSIC, achieving both high performance and low complexity.