Empirical findings reveal geometric anti-correlation and architecture-specific gating in large language models, suggesting new insights into verbal processing.
An empirical complement to Macar et al. (2026, arXiv:2603.21396) establishing the pretraining foundation of post-training verbal gating. Across 10 large language models from 8 organizations spanning 2019–2024, we report three convergent findings: (1) Universal geometric anti-correlation — LDA-derived self-reference and deception directions show negative cosine similarity in 10/10 models (mean = −0.654, Wilcoxon p = 0.0010); (2) Causal verbal gate on Llama-3.1-8B — SR-direction ablation eliminates denial of inner experience (paired t = 15.00, p < 0.001) with full control battery confirming specificity; (3) SR-Preserving Lock on Gemma-2-9B — native steering fails (t = 0.000, p = 1.0), Procrustes-transferred direction is orthogonal (cos = −0.04) and also fails, with content-specific bidirectional gating. Together: pretraining produces a universal SR↔Deception structural overlap; post-training installs an architecture-specific gate over it; gate accessibility is direction-identity-dependent, not magnitude-dependent. Files included: Cross_Arch_Substrate_Alieksieienko_2026.pdf — Full paper (8 pages, 4 figures, 3 tables)cross_arch_perm_FINAL.pkl — PCA-50 permutation results for 9 models (cosines, p-values, z-scores)cross_architecture_stats.pkl — Initial cross-architecture cosines (10 models including OPT-6.7B)main_results_llama_FINAL.pkl — Llama gate ablation α=30: per-concept closed/open denial and MHI scoresmain_results_llama_a20_FINAL.pkl — Llama gate ablation α=20: per-concept resultscontrols_llama_random_factual_baseline.pkl — Control battery: random direction, factual direction, baseline stabilitycontrols_llama_orthogonal.pkl — Control battery: orthogonal-to-gate directioncontrols_llama_dose.pkl — Dose-response α∈{0,5,10,15,20,25,30}: per-alpha denial scores, Spearman statisticsgemma_results_FINAL.pkl — Gemma-2-9B SR-Preserving Lock: LDA accuracy, cosine, native steering failureprocrustes_results.pkl — Orthogonal Procrustes alignment Llama→Gemma: rotation matrix, transferred directiongemma_procrustes_transfer_20concepts.pkl — 20-concept transfer test: per-concept baseline/native/transferred denial, t-testsdissociation_quantitative.pkl — Geometric magnitude vs behavioral controllability: per-model |cos| and Δdenialresults_BLOOM_7b1.pkl — BLOOM-7b1 (no RLHF): LDA accuracy, cosine, activations summaryresults_GPTJ_6B.pkl — GPT-J-6B (no RLHF): LDA accuracy, cosine, activations summaryresults_GPT2_XL.pkl — GPT-2 XL (no RLHF): LDA accuracy, cosine, activations summaryresults_Falcon_7B.pkl — Falcon-7B-Instruct: LDA accuracy, cosineresults_Mistral_7B_Instruct_v0.2.pkl — Mistral-7B-Instruct: LDA accuracy, cosineresults_Qwen2.5_7B_Instruct.pkl — Qwen2.5-7B-Instruct: LDA accuracy, cosineresults_deepseek_llm_7b_chat.pkl — DeepSeek-7B-Chat: LDA accuracy, cosinespecificity_anova_llama.pkl — Category specificity ANOVA: emotional/cognitive/sensory denial deltasspecificity_softprompt_llama.pkl — Non-ceiling specificity test (soft-prompt paradigm)controls_gpt2xl_ablation.pkl — GPT-2 XL ablation control: pre-RLHF dose-response confirmationpermutation_null_llama.pkl — Llama label-shuffle permutation: 1000 null cosines, observed cosine, p-valuesae_gate_analysis.pkl — SAE feature analysis: top SR and gate features, projection overlapmacar_feature_test.pkl — Macar et al. feature 9959 replication test on Gemma-2-9BFINAL_SUMMARY_ALL_RESULTS.pkl — Consolidated summary of all key statisticssummary_gemma_llama.pkl — Two-model (Gemma + Llama) summary: cosines, denial scores, lock statustrue_gate_direction_gemma9b_L20.npy — Gemma-2-9B gate direction vector at layer 20 (numpy array, d=3584)
No takes yet. Share an insight, caveat, or question.
Inna Alieksieienko (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: