We show that the large gap between effective rank (eRank) and TwoNN intrinsic dimension in LLM weight matrices is not evidence of manifold curvature, but of sub-manifold concatenation. When weight matrices are split along known functional boundaries — attention heads, Q/K/V segments — the eRank/TwoNN ratio drops by 1–2 orders of magnitude. Experiments on Qwen3.6-27B and DeepSeek V4 Flash (280B) demonstrate that per-head ratios converge to 4–9x across both models despite 10x difference in parameter count. Value heads in DeltaNet linear attention are nearly flat (ratio ≈ 2x). Deep MoE expert weights develop internal sub-manifold structure absent in shallow layers. Code and data are open-sourced.
Jin et al. (Sun,) studied this question.