Answers are grounded in this paper and cite their sources.
We ask whether task categories differ in the dimensionality of a pretrained language model’s internal representations and whether their relationship changes with depth. Last-input-token representations are extracted from Qwen2.5-1.5B for 200 prompts spanning fac-tual recall, arithmetic, reasoning, and coding. Representation geometry is summarized using participation ratio, covariance-spectrum entropy effective rank, principal-component vari-ance thresholds, and the TWO-NN local intrinsic-dimension estimator. The primary held-out analysis contains 12 prompts from two template families per category. For raw representa-tions, the reasoning-minus-recall participation-ratio contrast is positive at blocks 1–20 and negative at blocks 21–28. At block 24, participation ratio is 2.869 for factual recall and 1.397 for reasoning; after the final block and normalization, it is 2.235 and 2.126, respectively. The observations establish a descriptive depth-dependent relationship in these prompt sets, rather than a uniform ordering of task categories. Limited held-out family counts prevent reliable primary family-bootstrap intervals, and insufficient length overlap prevents a joint matched-length comparison. We report a bounded exploratory experiment with its analysis code and limitations.
No takes yet. Share an insight, caveat, or question.
Mitra Varun Gogulapati (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: