Decomposable interaction transformer with spectral adjustment for hyperspectral image and LiDAR classification

With the technological development of remote sensing, the volume of accessible multi-source remote sensing images is increasing sharply. Among them, the hyperspectral image (HSI) and light detection and ranging data (LiDAR) have attracted widespread attention, which are conducive to the land cover classification. However, modal difference and spectral variability pose challenges to the improvement of classification performance, which destroy the complementarity of heterogeneous spatial features and discriminability of spectral features. To this end, the decomposable interaction Transformer with spectral adjustment (DITWSA) is proposed. Firstly, the decomposable interactive spatial branch is designed to realize the heterogeneous spatial feature interaction by reducing the feature redundancy, which facilitates the spatial information exchange between modalities. Then, the adjustable multi-scale spectral branch is designed to optimize and adjust the spectral vector, which improves the discriminability of spectral features for ground objects. Finally, the obtained spatial and spectral features are jointly utilized for classification. The experimental results show that the classification performance of the proposed method outperforms that of other outstanding methods by 5.74% on average.