The Rotation-Retention Law: Knowledge Loss Is Proportional to Representational Rotation in Fine-Tuned Language Models. With EgoRA: Entropy-Governed Orthogonality Regularization for Adaptation

We introduce the Rotation-Retention Law: knowledge loss during fine-tuning is directly proportional to the mean rotation angle of attention head weight vectors. We validate this geometric relationship across 20 experimental conditions, three architectures (Llama, Mistral, Phi-3), and multiple model scales, and introduce EgoRA — a new entropy-governed orthogonality regularization method that substantially reduces catastrophic forgetting. I invite the research community to scrutinize these findings, replicate the experiments, and provide open feedback. ===Why does fine-tuning destroy pre-trained knowledge? We show the answer is geometric: knowledge loss is proportional to the mean rotation angle of attention head weight vectors between the pre-trained and adapted states. We formalize this as the Rotation-Retention Law: ΔM ∝ θ̄ = E[arccos(cossim(w_i^base, w_i^tuned))], where ΔM is the change in benchmark performance (MMLU) and θ̄ is the mean rotation angle across all attention heads. Across 20 experimental conditions with full benchmark validation—spanning three architectures (Llama 3.1 8B, Mistral 7B, Phi-3 Mini), three model scales (1B, 3B, 8B), two fine-tuning domains (instruction-following, medical), and three adaptation methods (LoRA, DoRA, EgoRA)—we find: (i) θ̄ > 5° predicts MMLU loss exceeding 3 pp in all high-rotation conditions; (ii) θ̄ < 3° predicts MMLU loss below 3 pp in all low-rotation conditions; (iii) the proportionality constant k = |ΔM|/θ̄ varies by regime and scale (0.22–0.96 pp/degree), with higher stability in the high-rotation regime. To establish causality, we introduce EgoRA (Entropy-Governed Orthogonality Regularization for Adaptation), which constrains rotation during training and reduces MMLU loss by up to 12.7× while reducing damaged heads from 56% to under 4%. We validate the law across three architectures, finding that while absolute rotation varies by 6× (3.6°–21.7°), the normalized rotation reduction from EgoRA is remarkably consistent (θ̄_norm = 0.42 ± 0.06), and higher rotation universally predicts greater MMLU loss. We propose θ̄_norm < 0.5 as a universal, architecture-independent certification threshold for knowledge-preserving fine-tuning. The Rotation-Retention Law reframes catastrophic forgetting as a geometric phenomenon and provides a falsifiable, architecture-agnostic diagnostic for fine-tuning quality.Submitted to SSRN and Zenodo on April 3, 2026. This is the initial public release for publication of the research findings.