Covariance Rank Collapse in Language Sequences: Three-Stage Empirical Evidence from Repetition, Preprocessing, and Polarity

We propose that structural collapse in language sequence dynamics is best characterizedas a reduction in the effective dimensionality of state transitions, whichwe term covariance rank collapse. Building on the reformulation of DirectionalEntropy Collapse (DEC) from directional alignment to dimensional reduction, wepresent a comprehensive empirical study across three experimental axes: repetitionrate, preprocessing methods, and semantic polarity. Using singular value decomposition(SVD) of embedding-based transition matrices, we show that (1) repetitionis the primary driver of rank collapse, (2) semantic convergence induces collapseeven without repetition, and (3) collapse is polarity-free and distinct from meretopical clustering. Explained variance ratio (EVR) analysis further confirms thatthe observed phenomenon corresponds to true rank collapse rather than singleaxisconcentration. These results establish covariance rank collapse as a robust,representation-agnostic indicator of reduced dynamical freedom in language systems.