Text-augmented patch–scene collaborative network for joint hyperspectral and LiDAR classification

Remote sensing (RS) image classification is an important technique for land-cover understanding and environmental monitoring. Combining hyperspectral imagery (HSI) with light detection and ranging (LiDAR) data has become a promising strategy for improving land-cover recognition. However, most existing approaches still focus on visual representation learning and make limited use of scene-level context and category-related semantic knowledge. Relying only on image features makes it difficult to associate visual patterns with land-cover semantics, especially for categories with similar appearances. Moreover, most methods focus on spectral–spatial feature extraction from local patches while overlooking potential scene dependencies among land-cover types. To address these limitations, this paper proposes a Text-Augmented Patch–Scene Collaborative Network (TAPSCN), a multilevel information fusion framework for joint HSI and LiDAR classification. TAPSCN integrates patch-level cross-modal cues, scene-level structural context, and category-level textual priors for scene-aware semantic classification. First, an Interactive Patch Fusion Module (IPFM) employs spectral-channel attention and geometric-spatial attention to accommodate the heterogeneous characteristics of HSI and LiDAR data, respectively. A two-stage asymmetric cross-modal mutual-guidance strategy further enhances collaborative representation learning across modalities, yielding robust local patch representations. Subsequently, a Global Scene Guidance Module (GSGM) models whole-image context through visual state-space modeling. By collaboratively optimizing local patch information and global scene semantics, GSGM improves the discrimination of complex scenes and easily confused classes. Finally, a Text Augmentation Alignment Module (TAAM) constructs semantic priors from class-level textual prompts. Through the joint optimization of strong vision–text alignment and weak semantic reconstruction, TAAM guides visual features to capture high-level semantic knowledge associated with land-cover classes, thereby enhancing their semantic discriminability. Extensive experiments on three real-world datasets demonstrate that TAPSCN achieves superior classification accuracy while maintaining moderate parameter complexity. The source code is publicly available at https://github.com/RogsXie/TAPSCN .