Selecting Related Knowledge via Efficient Channel Attention for Online Continual Learning

Continual learning aims to learn a sequence of tasks by leveraging the knowledge acquired in the past in an online-learning manner while being able to perform well on all previous tasks. This ability is crucial to the artificial intelligence (AI) system. Compared to the traditional learning pattern, continual learning is more suitable for most real-world and complex applicative scenarios. However, the current models usually learn a generic representation based on the class label on each task and an effective strategy is selected to avoid catastrophic forgetting. We postulate that selecting the related and useful parts only from the knowledge obtained to perform each task is more effective than utilizing the whole knowledge. Based on this fact, in this paper we propose a new framework, named Selecting Related Knowledge for Online Continual Learning (SRKOCL), which incorporates an additional efficient channel attention mechanism to pick the particularly related knowledge for every task. Our model also combines experience replay and knowledge distillation to circumvent catastrophic forgetting. Finally, extensive experiments are conducted on different benchmarks and the competitive experimental results demonstrate that our proposed SRKOCL is a promising approach against the state-of-the-art.