Valence-Aware Gradient Memory and Fractional Kernel Comparisons for LoRA-Based Language Model Unlearning

Selective machine unlearning aims to suppress targeted information while preserving retained and general knowledge. We investigate valence-aware gradient memory within low-rank adaptation (LoRA), separating retention-supporting and forgetting-related directions. Earlier experiments used Gamma-scaled exponential moving averages rather than genuine fractional dynamics; we preserve that distinction and introduce a normalized finite-history Grünwald–Letnikov (GL) fractional-integral accumulator with algebraically decaying coefficients. The study progresses from synthetic transformer optimization and pretrained text classification to causal language model experiments, including a new controlled evaluation on TOFU Forget10 using Qwen2.5-0.5B-Instruct. A SciQ- and WikiText-regularized teacher reduces reference-answer NLL on original and paraphrased TOFU questions while exhibiting a small World Facts penalty in a separate audit. From this fixed teacher, four unlearning methods were compared for 120 updates over three random seeds. Dual GL yields the largest paraphrased forget-set NLL increase (2.169±0.331), exceeding dual EMA (1.964±0.250), but incurs greater retain-set (1.043±0.207 versus 0.914±0.147) and World Facts (1.333±0.645 versus 1.153±0.567) degradation. Exact paired sign-flip tests do not establish statistical significance (p=0.25 for GL versus EMA). The stronger suppression achieved with fractional memory is accompanied by greater disruption of retained and general factual knowledge, suggesting a shift in model behavior rather than more selective forgetting. Whether the targeted information becomes genuinely inaccessible rather than merely less likely to be reproduced remains unresolved.