What does this research mean for the field?

Direct Preference Optimization (DPO) in large language models functions primarily by suppressing rejected tokens rather than promoting correct ones, effectively teaching models what is false rather than what is true. Novelty: ClaimNovelty.NOVEL_FINDING. Consensus alignment: ConsensusAlignment.CHALLENGES_CONSENSUS.

What question did this study set out to answer?

This research aims to investigate how transformers utilize DPO and its effects on factual knowledge suppression.

May 17, 2026Open Access

Project Aletheia V5: The Suppression Paradox — DPO as Inhibitor, Not Promoter

Key Points

This research aims to investigate how transformers utilize DPO and its effects on factual knowledge suppression.
Conducted a systematic 129-phase investigation using GPT-2 and scaling to Qwen2.5-14B.
Established seven laws, five theorems, and three principles governing the DPO's mechanisms.
Analyzed statistical significance and scaling of learning rates and model robustness.
DPO suppresses rejected tokens with 100% reliability but promotes correct ones only 73% of the time.
L23's DPO edit is statistically significant (z=4.84, p<0.001), achieving performance with fewer parameters.
Numerical token embeddings show a fundamental limitation of preference optimization with a clustering cosine of 0.73.

Abstract

This paper presents Project Aletheia, a systematic 129-phase investigation of LLM hallucination through the lens of condensed matter physics. Using GPT-2 (124M) as a "particle accelerator" and scaling to Qwen2.5-14B, I establish seven fundamental laws, five theorems, and three new principles governing how transformers suppress factual knowledge. V5 New Discoveries (Phases 114–129, Seasons 27–28): The DPO Suppression Theorem: DPO primarily works by suppressing rejected tokens (100% reliability), not promoting correct ones (73%). DPO does not teach models what is true — it teaches them what is false. Single-Layer Sufficiency: Only L23's DPO edit is statistically significant (z=4.84, p<0.001). L23 alone with 2× lr matches L22+L23 with 50% fewer parameters. Phase Boundary Scaling: The critical DPO learning rate scales as lr* ~ N0.83. Larger models are more robust to alignment hyperparameters. Numerical Token Immunity: Numerical token embeddings are 9× more clustered (cosine 0.73 vs. 0.08) with zero baseline confidence, rendering DPO effect literally zero — a fundamental limitation of preference optimization. Previously established in V1–V4: Grammatical Suppression of Facts (GSF): 70% of facts suppressed by final layers; L9H6 (+927) is the top suppressor Code Mode Switch: Any symbol prefix (#, //, --) triggers a mode transition reducing GSF The 14B Singularity: Qwen2.5-14B achieves 100% factual accuracy with Code Mode — zero external knowledge, zero fine-tuning Aletheia Constant (αA ≈ 0.95): Universal across architecture, language, and temperature Phases 1–113 from V4 fully preserved Acknowledgments This research was conducted entirely independently, without institutional affiliation or corporate funding. The author currently faces financial constraints that make it increasingly difficult to maintain subscriptions to AI services essential for this line of research. To sustain and improve the quality of future work, the author is actively seeking community sponsorship. Details are available at https://github.com/sponsors/hafufu-stack.

Read Full Paperexternally

AI에게 질문

Bookmark

View Full Paper