A Unified Knowledge Management Framework for Continual Learning and Machine Unlearning in Large Language Models

Large language models (LLMs) are increasingly deployed as information systems that evolve over time, where managing internal knowledge—acquisition, retention, and removal—becomes essential. In practice, these processes are primarily realized through continual learning and machine unlearning mechanisms. Despite this, these two mechanisms are often studied in isolation, limiting both interpretability and controllability. In this work, we present a parameter-efficient knowledge management framework where continual learning and machine unlearning—despite employing distinct task-specific objectives—are integrated through a shared retention-controlled parameter evolution mechanism. We ground these structural constraints in a drift-aware design principle: under a model smoothness assumption, we establish a formal upper bound showing that Kullback–Leibler (KL) divergence on retained knowledge is controlled by the magnitude and direction of parameter updates, providing a principled rationale for combining Low-Rank Adaptation (LoRA) freezing, sparse masking, and orthogonal gradient projection into a unified constraint system. Experiments on the Task of Fictitious Unlearning (TOFU) benchmark and real-world benchmarks demonstrate effective knowledge acquisition, selective removal, and robust retention across sequential tasks with strong overall performance and stability. This work provides a practical parameter-efficient recipe and a drift-aware design principle validated on controlled interleaved benchmarks, offering insights toward reliable knowledge management in evolving deployment scenarios.