A lightweight network of groupwise separable convolution and vision transformer for hyperspectral image classification

Hyperspectral image (HSI) data consists of images with numerous contiguous spectral bands and promotes the extensive applications in the field of remote sensing. Recent approaches based on Vision Transformer (ViT) have achieved remarkable performance in HSI classification tasks due to their ability to extract global spatial features and model long range dependencies. However, ViT has complex network structure, training is challenging and lacks adequate consideration of local spatial and spectral receptive fields in hyperspectral data. To solve the problems above, we propose a lightweight network model known as the Groupwise Separable Convolution and Vision Transformer (GSCViT). Firstly, we introduce a parameter-free attention for spectral calibration (SC). Then, we meticulously design a novel convolutional approach named Groupwise Separable Convolution (GSC), which greatly reduce the number of convolutional kernel parameters and effectively capture local spatial-spectral information in HSI data. In addition, we employ Groupwise Separable Multi-Head Self-Attention (GSSA) to replace the traditional Multi-Head Self-Attention (MSA) in ViT, thus can simultaneously attend to local and global spatial features in HSI with lower computational burden. Experiments on two benchmark HSI datasets demonstrate that our GSCViT model achieves excellent accuracy with relatively small training samples and outperforms some existing HSI classification algorithms.