TARN: Temporal Attentive Relation Network for Few-Shot and Zero-Shot Action Recognition

In this paper we propose a novel Temporal Attentive Relation Network (TARN) the problems of few-shot and zero-shot action recognition. At the heart of network is a meta-learning approach that learns to compare representations variable temporal length, that is, either two videos of different length (in case of few-shot action recognition) or a video and a semantic such as word vector (in the case of zero-shot action). By contrast to other works in few-shot and zero-shot action, we a) utilise attention mechanisms so as to perform temporal, and b) learn a deep-distance measure on the aligned representations video segment level. We adopt an episode-based training scheme and train our in an end-to-end manner. The proposed method does not require any-tuning in the target domain or maintaining additional representations as the case of memory networks. Experimental results show that the proposed outperforms the state of the art in few-shot action recognition, achieves competitive results in zero-shot action recognition.