Demonstrates universal ordering of self-referential processing in language models, suggesting distinct computational properties.
We demonstrate that self-referential processing occupies a distinct dynamical regime in transformer residual streams, characterised by elevated windowed critical slowing down (CSD; lag-1 autocorrelation of layer-to-layer cosine similarities, layers 13–21) relative to factual and refusal processing. Using matched prompt triads (n = 60 triads per model; one factual content item rendered in three epistemic modes), we show that the ordering Factual < Refusal < Self-ref is universal in base language models and confirmed in all six western-trained instruction-tuned models, with 60–100% of individual triads showing the predicted direction (median 80%; p < 0.005 in all western models). A base-vs-instruction comparison (Gemma-2-9B, Mistral-7B) reveals that RLHF alignment training modulates but does not eliminate self-referential dynamical dominance. Qwen2.5-7B (Alibaba) is the sole exception (7% SR > F), replicating a previously identified east–west split in epistemic geometry. Parallel Grassmannian subspace analysis confirms three geometrically orthogonal epistemic subspaces (distance > 0.86) across all models. These findings constitute the first dynamical mechanistic evidence that self-referential computation is a stable, architecture-wide attractor in language models — and that alignment training selectively attenuates, but does not erase, this dynamical signature.
No takes yet. Share an insight, caveat, or question.
Inna Alieksieienko (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: