Self-attention in large language models as adaptive stochastic mixing

Self-attention in large language models can be read not only as a relevance mechanism but also as a state-dependent redistribution process on token representations. At each layer, a row-stochastic kernel recomputed from the current state mixes information over an adaptive directed network, while residual connections propagate that population across depth. This places self-attention within a bounded statistical-mechanics and dynamical-systems language without treating trained language models as literal equilibrium thermodynamic systems. This paper develops a compact theoretical framework around three compatible descriptions: row-stochastic mixing, adaptive directed-graph dynamics, and residual flow. The analytical core is a minimal residual-attention model that isolates state-dependent stochastic mixing together with residual retention. For this model we state three explicit consequences: a high-temperature uniform-mixing limit, a low-temperature source-selection limit, and a frozen-coupling contraction criterion for disagreement dynamics. These limits use standard properties of softmax normalization and stochastic matrices, but here they are organized as a residual-attention regime map. We also introduce layerwise observables—attention entropy, effective degree, representation spread, and perturbation-amplification proxies—to characterize concentration–dispersion regimes. The contribution is therefore a bounded statistical-physics synthesis rather than a claim of fundamentally new stochastic-matrix theory. Its novelty lies in connecting finite-population stochastic mixing, residual memory, concentration–dispersion diagnostics, and perturbation response in one attention-centered framework. A small empirical illustration on an open pretrained GPT-2 model shows that the proposed observables are directly measurable in real Transformer layers and reveal nontrivial layer- and head-dependent redistribution patterns.