Marginal-Preserving Support Interventions in Synthetic Transformer-Block Distillation

A Transformer hidden vector can already encode substantial context from earlier layers. How much does its cross-position arrangement matter when a compressed block learns from synthetic hidden sequences? We study this question with Q_pos, a support intervention that independently permutes donor rows at each position while preserving the exact finite empirical multiset of complete hidden vectors. Students learn frozen block residual updates from Native or Q_pos synthetic inputs, with fresh Teacher labels and a fixed paired training procedure. Three separately registered campaigns reveal configuration-dependent effects. At GPT-2 block 5, Q_pos produces material losses on both value and history-response fidelity: mean normalized-error penalties are 0.063807 and 0.029738. GPT-2 block 9 meets its registered 5%-of-Native-recovery preservation rule, and Pythia-160M block 5 meets qualified preservation under absolute margins. Exploratory cross-support evaluation locates the GPT-2 block-5 deficit on independent Native synthetic inputs as well: Native- and Q_pos-trained Students score 0.102917 and 0.156757 on that common support, while their ranking reverses on Q_pos support. The study establishes a controlled empirical failure of marginal sufficiency for one fixed learning configuration, alongside preservation cases, and characterizes the resulting support-conditioned approximations. It shows why evaluating learned operators on common supports is essential to understanding synthetic distillation. Associated software and numerical results: LatentBridge version 0.1.0 (commit b11491d4d729ac13870e4e443796e57509e75674). Original software is MIT licensed. This manuscript and its source files are CC BY 4.0. This is a preprint, not a peer-reviewed publication. Generative AI tools, including OpenAI Codex and other language-model assistants, substantially assisted research planning, experimental and analysis code development, literature organization, and manuscript drafting and revision. The author retains responsibility for the scientific content, verification of results and references, and the final manuscript.