Internal State Geometry is Measurable in Small Open Language Models: A Cross-Family Survey

I report a survey of dense activation observation on twelve open language models spanning five architecture families (BLOOM, BLOOMZ, Phi-2, Pythia, Qwen2. 5, InternLM-2) and parameter counts from 124M to 7. 6B. Every model in the qualified set produces an observable internal state structure under a fixed-budget probing protocol: coverage in the band 90. 6% to 97. 05%, total channel count from 523k to 1. 54M, hooked-module count from 147 to 253. Three cross-family patterns are reported: module-count-to-parameter ratio varies by an order of magnitude between architectures (23 to 84 modules per billion parameters) ; channel count scales sub-linearly with parameter count C ~ Nₚ⁰. 63 on this sample; the linear-to-norm module ratio is roughly 5: 1 across families. Small open LMs are not opaque substrates; they are dense, observable, and comparable.