Answers are grounded in this paper and cite their sources.
Demonstrates improved performance in class-incremental learning, suggesting a new method for knowledge retention.
Continual learning aims to enable models to incrementally acquire knowledge from non-stationary data streams, yet it remains hindered by catastrophic forgetting, where performance on previously learned tasks deteriorates when adapting to new ones. While pretrained foundation models mitigate this issue to some extent through strong generalization, naive incremental fine-tuning inevitably leads to knowledge degradation. In this work, we advance continual learning with two complementary contributions. First, we propose a knowledge consolidation strategy that achieves representation consolidation through model backbone merging, allowing continual task-wise training and subsequently merging task-specific parameters into a unified representation, thereby improving adaptability while mitigating forgetting. Our merging step is grounded in recent theory on task arithmetic, which guarantees generalization when task vectors are aligned in parameter space — a condition our sequential training satisfies by construction. Second, recognizing that the merged backbone may introduce mismatches with existing classifiers, we introduce a classifier alignment mechanism based on Natural Evolution Strategies, which evolves classifier parameters to achieve effective alignment with the consolidated backbone. Together, these contributions provide a principled framework that simultaneously consolidates representations and aligns classifiers, offering a robust solution to continual learning challenges. Experiments on seven class-incremental benchmarks show that our method — Knowledge Consolidation with Evolutionary Alignment (KCEA) — is competitive with state-of-the-art replay-free methods on final accuracy and sets a new state of the art on accumulated accuracy. Code is publicly available at https://github.com/tungts1101/KCEA.git .
No takes yet. Share an insight, caveat, or question.
Tran et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: