TA-MSA: A Fine-Tuning Framework for Few-Shot Remote Sensing Scene Classification

Existing few-shot remote sensing scene classification (FS-RSSC) works primarily follow the meta-learning paradigm, which meta-trains a model on an auxiliary dataset before adapting it to target FS-RSSC tasks. To ensure good performance, the auxiliary dataset should share similar distributions with the target tasks. However, acquiring such an auxiliary dataset is difficult and economically costly in real-world FS-RSSC applications. To address this issue, we aim to handle FS-RSSC tasks by directly fine-tuning a general pre-trained model, eliminating the need for an auxiliary dataset related to the target tasks. In this paper, we propose a novel fine-tuning framework, named TA-MSA, which consists of a Task-Adaptive (TA) fine-tuning strategy and a Multi-level Spatial feature Aggregation (MSA) module. The TA fine-tuning strategy is composed of two components: (1) a layer-specific optimizer that alleviates distribution shifts between the pre-trained and target remote sensing datasets, and (2) a task-specific training scheme designed to accommodate variations in discriminative features across different FS-RSSC tasks. Additionally, to suppress the negative effect of the cluttered backgrounds and enhance the spatial features of true discriminative regions, the MSA module extracts multi-level spatially important features using trainable spatial templates for classification. Experimental analysis demonstrates the superiority of the proposed TA-MSA framework. On three FS-RSSC benchmarks (NWPU-RESISC45, UC Merced LandUse, and WHU-RS19), our TA-MSA framework outperforms many state-of-the-art methods, achieving an average classification accuracy of 76.78% in the 5-way 1-shot setting and 91.89% in the 5-way 5-shot setting.