An Equal-Byte Non-Inferiority Protocol for Storage-Aware, Reproducible Evaluation of Replay-Based Continual Learning

Replay-based continual learners are commonly evaluated at a fixed item count, but deployment hardware meters memory in bytes, and methods storing logits, feature vectors, or compressed codes occupy different byte footprints at an equal item count. The equal-item convention therefore does not answer the deployment question. We propose an equal-byte non-inferiority protocol that holds total stored bytes constant across competitors, selects the strongest of a pre-registered eight-method baseline pool at each budget, and reports a decision at a registered margin of ΔNI=0.05 under Holm–Bonferroni correction. We demonstrate it on Split-CIFAR-100 and on an eight-task family-grouped sequence of fifty curated Abstraction and Reasoning Corpus puzzles, using a two-tier consolidation memory with a byte-budget knob, at ten seeds per cell. Under the pre-registered rule, the two-tier scheme fails non-inferiority at all four budgets on both benchmarks, the strongest per-buffer baseline exceeds the best two-tier cell at every budget, and the outcome is robust to task ordering on the abstract-reasoning benchmark. An interpretability analysis links the outcome to the compressed tier’s attention share and prototype similarity. The contribution is the protocol; the demonstration returns a clean negative decision on one specific candidate.