Relational Synthesis Pretraining: Harmonizing Knowledge via Cognitive-Link Modeling

While augmenting pretraining corpora with retrieved neighbors has shown promise, the prevailing paradigm of Large Language Model (LLM) optimization demands a delicate equilibrium between data integrity and thematic variety. Conventional methods that aggregate near-identical documents often inadvertently facilitate "surface-level memorization," where models exploit local patterns rather than internalizing deep semantic structures. Such redundant signals can compromise the model's inductive bias. To circumvent these pitfalls, we introduce a transformative pretraining framework centered on functional relevance rather than lexical similarity. Our approach simulates human-like inquiry by generating multi-faceted interrogatives from anchor documents to probe the broader corpus. By selectively integrating these logically connected (yet non-redundant) segments into unified training trajectories, we reorganize the existing token space without inflating the total computational budget. Empirical evaluations indicate that this relational restructuring significantly bolsters LLM performance across diverse cognitive benchmarks.