What question did this study set out to answer?

This research aims to enhance knowledge transfer in continual learning by leveraging information flow within neural networks.

May 24, 2026Open Access

Leveraging Information Flow for Knowledge Transfer in Continual Learning

JAJoshua AndleUniversity of Maine APAli PayaniCisco Systems (United States)SYSalimeh Yasaei-SekehSan Diego State University

Key Points

This research aims to enhance knowledge transfer in continual learning by leveraging information flow within neural networks.
Investigated the effects of pruning methods in continual learning using subnetworks for different sequential tasks.
Leveraged information flow to determine optimal weight sharing strategies for forward knowledge transfer.
Conducted experiments on three continual learning datasets using ResNet-18 and VGG-16 architectures.
Showed that optimal weight sharing based on information flow improved forward knowledge transfer accuracy.
Prevented catastrophic forgetting and negative backward knowledge transfer through selective weight sharing.
Compared accuracies across benchmark methods, demonstrating significant improvements in task performance.

Abstract

Continual Learning (CL) has proven an effective method of avoiding Catastrophic Forgetting (CF) when sequentially training neural networks. This improves network efficiency while facilitating the Knowledge Transfer (KT) between tasks. CL serves as an ideal setting for studying KT and network behavior. In particular, pruning methods for CL train subnetworks to handle the sequential tasks which allows us to take a structured approach to investigating KT. This subnetwork approach allows for the promotion of Forward KT (FKT) and complete prevention of negative Backward KT (BKT) and CF. Understanding which weights to share between tasks is crucial for FKT as sharing all weights can worsen accuracy. This paper demonstrates how the information flow (IF) within the network can reflect task similarity and usefulness, informing optimal sharing decisions to improve FKT between diverse tasks. We leverage IF to determine which weights to share to promote FKT. By doing this, we match the optimal sharing decisions for each implemented dataset, and compare the resulting accuracies against several benchmark methods which emphasize promoting FKT and BKT. Experiments are run on three CL datasets designed to emphasize variation in task complexity and similarity, using both ResNet-18 and VGG-16.

Ask AI

Helpful

Bookmark

View Full Paper