Sencelium: a recurrent language model with self-monitoring signals and a growing memory that is written at inference

Recurrent alternatives to the Transformer compress the past into a fixed-size state, freeze their knowledge when training ends, and give the model no view of its own computation. We describe Sencelium, a recurrent language model built to address these limits together. Each block pairs a content-addressed gated-delta path with a content-blind moving-average path whose retention the delta path modulates; nineteen introspective signals about the block's own statistics feed back into its retention; a memory pool that starts empty grows by a learned spawn/join/merge policy and is written by the same code during training and at inference; a short inner-voice loop reads the memory before the output; and a slot-memory residual summarises the running context. We train Sencelium at 30M, 65M and 125M parameters with compute-optimal token budgets against a vanilla Transformer of matched size, data and window. Sencelium trails the Transformer on perplexity at every scale, by 18.7%, 11.5% and 9.2%, and the gap narrows with size. Beyond the trained length its perplexity keeps falling up to 16 times the window while that of a rotary Transformer without length extension rises, but a Transformer evaluated with a sliding window remains better. We then measure what some of the components do (the memory, carried state and the essence residual; the signals, the nucleus and the inner voice are not ablated at these scales). The memory is load-bearing but its benefit is structural: replacing its content by random, permuted or collapsed vectors changes perplexity by under 0.7 points, whereas emptying it costs 4 to 24 points, and a fact written at inference is not used better than an unrelated one. Carried recurrent state lowers evaluation perplexity by up to 17% by removing a cold start, and the essence residual acts mostly as a recency prior. We describe the verification protocol (code audits found three causal leaks and one data-ordering leak, and perturbation tests confirm the fixes), report mechanisms that did not help, and release code, evaluation scripts and checkpoints.