ICTNet: Interactive Convolution and Transformer Network for Hyperspectral Image Classification

The classification of a hyperspectral image (HSI) plays a critical role and serves as the foundation for many related applications. The combination of convolutional neural networks (CNNs) and transformers has shown promising performance in HSI classification by using the advantages of the two networks. However, existing hybrid models often offer limited feature interaction and fusion between the two branches. Here, a novel interactive convolution and transformer network (ICTNet) for HSI classification is proposed. Specifically, raw HSI is first fed into a multi-scale feature-enhancement module, where convolution operations with varying kernel sizes are used to extract multi-scale features, and shuffle attention is used to further enhance feature representation. Enhanced features are then processed by a dual-branch CNN and transformer fusion module to leverage local information and long-range dependencies. Additionally, the feature interaction and fusion module is proposed to facilitate dynamic interaction and fusion of features during extraction and propagation between the two branches, enhancing the diversity and interactivity of the features. Extensive experimental results on three real HSI data sets demonstrate that the proposed ICTNet outperforms state-of-the-art HSI classification methods.