Robust Few-Shot Classification via Bilevel Knowledge Distillation

In this paper, we propose a few-shot learning method for robust image classification using knowledge distillation. Existing benchmark datasets for few-shot learning only consists of clean images. These datasets have the problem of not being able to reflect the real-world degradations such as noise and corruptions. In addition, there are still few studies on few-shot learning for robust image classification in corruption. Therefore, in this study, we first propose four novel datasets, Mini-ImageNet-C, CUB-200-C, CIFAR-FS-C and FGVC-Aircraft-C for evaluating the robustness of few-shot learning algorithms. As the baseline few-shot learning method, we employ the most representative meta-learning approach, especially Model-Agnostic Meta-Learning (MAML). Afterwards, we incorporate knowledge distillation (KD) into MAML to distill corruption robustness from the large teacher model to the small student model where KD is performed in both inner-loop and outer-loop of MAML. Our 'Bilevel KD' allows the student models to achieve better performance while maintaining low memory usage. At meta-test stage, experiments showed that in all cases, our method performed significantly better than the baseline.