Empirical evidence shows LLMs process structured formats independently of semantics, suggesting new neural circuit implications.
This paper provides empirical evidence that LLMs process structured formats (JSON/XML) through a dedicated "structural motor circuit" that is functionally independent of semantic processing. Four categories of experiments — three behavioral (DeepSeek-V3) and one internal attention analysis (Llama-3.3-70B, Qwen2.5-72B) — demonstrate: (1) structure-content double dissociation, (2) noise-robust silent recovery with 0% upper-layer awareness, (3) directional switching inertia correlated with closing-tag explicitness, and (4) structural attention heads concentrated in shallow layers (top 20%), separated from deep semantic layers. Cross-format (JSON vs XML) and cross-model (Meta vs Alibaba) consistency confirms this as an emergent property of the Transformer architecture, not a training-specific artifact. The structural circuit functions analogously to procedural memory in biological systems.
No takes yet. Share an insight, caveat, or question.
Jin et al. (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: