What question did this study set out to answer?

This research examines whether fine-tuning a language model on contradictory information erases prior knowledge or merely masks it.

June 18, 2026Open Access

Compete, Don't Erase: Contradictory Fine-Tuning Masks Rather Than Deletes Knowledge

View Full Paper

Ask AI

Bookmark

This research examines whether fine-tuning a language model on contradictory information erases prior knowledge or merely masks it.
Analyzed log-probabilities of old and new knowledge during fine-tuning on three language models (Qwen2.5-1.5B/3B, Llama-3.2-3B).
Conducted localized weight edits on a different model (GPT-2 XL) to differentiate erasure from masking.
Created and released synthetic-fact generators, training scripts, and logit trajectories for evaluation.
Contradictory fine-tuning maintains old knowledge within ~1-1.5 nats of pre-contradiction reference, with no facts erased.
Localized weight edits result in a significant collapse of the log-probability for the old value (~20 nats), indicating erasure in that case.
A context shift allows for partial recovery of old knowledge, suggesting rapid reacquisition akin to extinction effects in behavioral neuroscience.

No takes yet. Share an insight, caveat, or question.

Randomized trial demonstrates that fine-tuning on contradictory facts leaves prior knowledge intact, indicating a masking effect.

Tomas Pødenphant Lund (2026) studied this question.

View Full Paper

Ask AI

Bookmark