Answers are grounded in this paper and cite their sources.
Mechanistic interpretability study reveals attention head behavior and semantics in transformer models, suggesting a fixed operator model.
This Pre-Print presents a mechanistic interpretability study investigating how transformer language models (specifically GPT-2-small) route and represent semantic concepts. The research empirically evaluates two related theories of transformer computation: The Forward Proportion Hypothesis: The theory that semantic behavior is conditioned on the activation trajectory rather than solely the current state. The Dynamic Computational Pathway Hypothesis: The theory that meaning emerges dynamically through context-sensitive interaction topologies between attention heads. Through a two-iteration, multi-phase experimental pipeline involving curated prompts across six semantic categories, we subject these hypotheses to rigorous falsification protocols including activation patching, pathway perturbation, and directional decomposition. Key Findings: Falsification of Dynamic Pathways: We demonstrate that individual attention heads do not dynamically rewire across contexts. Furthermore, targeted disruption of direct head-to-head attention pathways yields only $3.5%$ of the effect of full component ablation, indicating that direct inter-head pathways are not causally significant for semantic formation. The Combinatorial Recruitment Model: The empirical data strongly supports a model where attention heads act as fixed-behavior operators communicating indirectly via a shared residual stream. Contextual semantics emerge from the combinatorial selection (the "team roster") of active components, aligning with recent dictionary learning literature on feature superposition. Irreducible Architectural Variance: Even when using advanced, unembedding-aware metrics (e.g., unembedding-projected distance, post-LayerNorm similarity), $27.3%$ of the semantic divergence between convergent prompt trajectories remains unpredictable from single-position residual stream similarity. Directional decomposition reveals that this difference energy is moderately concentrated, highlighting a fundamental architectural limit in state-based prediction caused by distributed, nonlinear, multi-position processing. This manuscript details the complete methodology, mathematical formulations, ablation tables, and structural limitations of the study.
No takes yet. Share an insight, caveat, or question.
Pushp Kharat (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: