We introduce the Rotation-Retention Law: knowledge loss during fine-tuning is directly proportional to the mean rotation angle of attention head weight vectors. We validate this geometric relationship across 20 experimental conditions, three architectures (Llama, Mistral, Phi-3), and multiple model scales, and introduce EgoRA — a new entropy-governed orthogonality regularization method that substantially reduces catastrophic forgetting. I invite the research community to scrutinize these findings, replicate the experiments, and provide open feedback. ===Why does fine-tuning destroy pre-trained knowledge? We show the answer is geometric: knowledge loss is proportional to the mean rotation angle of attention head weight vectors between the pre-trained and adapted states. We formalize this as the Rotation-Retention Law: ΔM ∝ θ̄ = Earccos (cossim (wᵢᵇase, wᵢᵗuned) ), where ΔM is the change in benchmark performance (MMLU) and θ̄ is the mean rotation angle across all attention heads. Across 20 experimental conditions with full benchmark validation—spanning three architectures (Llama 3. 1 8B, Mistral 7B, Phi-3 Mini), three model scales (1B, 3B, 8B), two fine-tuning domains (instruction-following, medical), and three adaptation methods (LoRA, DoRA, EgoRA) —we find: (i) θ̄ > 5° predicts MMLU loss exceeding 3 pp in all high-rotation conditions; (ii) θ̄ < 3° predicts MMLU loss below 3 pp in all low-rotation conditions; (iii) the proportionality constant k = |ΔM|/θ̄ varies by regime and scale (0. 22–0. 96 pp/degree), with higher stability in the high-rotation regime. To establish causality, we introduce EgoRA (Entropy-Governed Orthogonality Regularization for Adaptation), which constrains rotation during training and reduces MMLU loss by up to 12. 7× while reducing damaged heads from 56% to under 4%. We validate the law across three architectures, finding that while absolute rotation varies by 6× (3. 6°–21. 7°), the normalized rotation reduction from EgoRA is remarkably consistent (θ̄ₙorm = 0. 42 ± 0. 06), and higher rotation universally predicts greater MMLU loss. We propose θ̄ₙorm < 0. 5 as a universal, architecture-independent certification threshold for knowledge-preserving fine-tuning. The Rotation-Retention Law reframes catastrophic forgetting as a geometric phenomenon and provides a falsifiable, architecture-agnostic diagnostic for fine-tuning quality. Submitted to SSRN and Zenodo on April 3, 2026. This is the initial public release for publication of the research findings. © 2026 Mark Dillerop. All rights reserved. This preprint is licensed under a Creative Commons Attribution 4. 0 International License (CC BY 4. 0). The methods described in this work are subject to U. S. Provisional Patent Application No. 64/024, 742.
Mark Dillerop (Fri,) studied this question.