TAKD: Target-Aware Knowledge Distillation for Remote Sensing Scene Classification

Remote sensing (RS) scene classification based on deep neural networks (DNNs) has recently drawn remarkable attention. However, the DNNs contain a great number of parameters and require a huge amount of computational costs, which are hard to deploy on edge devices such as onboard embedded systems. To address this issue, in this paper, we propose a target-aware knowledge distillation (TAKD) method for RS scene classification. By considering the characteristics among the target and background regions of the RS images, the TAKD can adaptively distill the knowledge from the teacher model to create a lightweight student model. Specifically, we first introduce a target extraction module that utilizes heatmaps to highlight target regions on the teacher's feature maps. Next, we propose an adaptive fusion module that aggregates these heatmaps to capture objects with varying scales. Finally, we design a target-aware loss that enables the transfer of knowledge in the target regions from the teacher model to the student model, greatly reducing background disturbance. Our distillation scheme that does not require extra learning parameters is both simple and effective, significantly improving the accuracy of the student model without any additional computational or resource costs. Our experiments on three benchmark datasets demonstrate that our proposed TAKD outperforms the existing state-of-the-art distillation methods.