What question did this study set out to answer?

This research aims to improve parameter selection in continual learning to optimize task adaptation while preserving model capacity.

March 16, 2026Open Access

Engram: Parameter Compression with Modulator Signals in Fixed-Capacity Continual Learning

Key Points

This research aims to improve parameter selection in continual learning to optimize task adaptation while preserving model capacity.
Introduced the Engram method for parameter selection based on modulator signals from training data.
Compared Engram with PackNet and its variants on P-MNIST and Split-MNIST datasets.
Focused on selecting parameters based on importance signals during training.
Engram achieved a mean accuracy of 81.98 on P-MNIST compared to 67.89 for PackNet.
On high-cap Split-MNIST, all methods reached nearly saturated accuracy.
Engram requires larger training-state memory but maintains small increases in deployment storage.

Abstract

Continual learning under a fixed parameter budget requires deciding which parametersshould be preserved and which can still be changed as new tasks arrive. Methods such asPackNet mainly rely on weight magnitude when selecting parameters, which can miss smallbut important parameters. We propose the Engram method, which uses modulator signalsconstructed from loss, logits, and gradients during ordinary training to identify importantparameters and uses phase swap to compress task-related information distributed across thebackbone into a selected parameter subset. In a setting where task information is given, wecompare Engram with PackNet and two variants that differ only in the selection score onP-MNIST and Split-MNIST under similar realized occupancy conditions. On P-MNIST,Engram achieves higher after10 mean accuracy, with the largest gain in the low-cap condition,reaching 81.98 versus 67.89 for PackNet at Uglob ≈ 0.0482. On high-cap Split-MNIST, allmethods reach nearly saturated accuracy. These gains require larger additional training-statememory and longer runtime, but under the same sparse export rule the increase in totaldeployment storage is relatively small at 5.7%

Engram: Parameter Compression with Modulator Signals in Fixed-Capacity Continual Learning

Key Points

Abstract

Cite This Study