Deep Learning Model With Fine-Tuning for Generalized Few-Shot Activity Recognition

The problem we focused on in this article is sensor-based generalized few-shot activity recognition. In this problem, each of the predefined activity classes (i.e., base classes) has substantial training instances, while each of the new activity classes (i.e., novel classes) just has a few training instances. Both the base and the novel classes need to be recognized. Currently, just a few works focus on this problem, and no formal statement of the problem is provided. In this article, we provide a formal definition of the problem, and propose a method to address it. In the proposed method, adopting the strategy of fine-tuning deep learning models, a deep learning model is first learned with the base-class training instances, and then fine-tuned with resampled training instances from both the base and the novel classes. We evaluate our method with three publicly available datasets on 1-shot, 5-shot, and 10-shot learning tasks. The results on the evaluation metric of harmonic mean of the average per-class accuracy for the base classes and that for the novel classes show that, our method could outperform state-of-the-art methods. In addition, the time and resource cost of our method is moderate.