What question did this study set out to answer?

The research aims to elucidate the roles of experts within a Mixture-of-Experts language model, focusing on a reflective worldview register.

July 5, 2026Open Access

A Single-Expert Readout of a Reflective Worldview Register in a Mixture-of-Experts Language Model

Key Points

The research aims to elucidate the roles of experts within a Mixture-of-Experts language model, focusing on a reflective worldview register.
Characterization of Expert 114 in Qwen3.5-35B-A3B model as a readout of the reflective worldview register.
Conducting various tests to separate register generations from controls, including held-out and cross-model tests.
Employing a prompt-independent auto-interpreter to evaluate the model's performance across abstract and philosophical contexts.
Expert 114 produced reflective worldview register generations with a Cohen's d = 3.88 from lexically matched controls.
The auto-interpreter achieved an AUC of 0.94 in recovering the same register, displaying its capability beyond self-reference.
The model-specific nature of Expert 114 indicates distinct roles in the Qwen3.5-35B-A3B model.

Abstract

A Single-Expert Readout of a Reflective Worldview Register in a Mixture-of-Experts Language Model. Preprint, version 1.2.5 (June 2026). Peer review pending. Mixture-of-experts (MoE) routing emits a discrete, per-token record of which experts fire, a signal unusually legible for interpretability, yet single experts are rarely tied to a specific functional role. We study a reflective worldview register: generated language that sustains an interpretive stance toward meaning, belief, value, existence, or the interiority of a target. Examination is the process we use to elicit this stance; the target can be the model, another entity, a natural object, or an abstract subject. In Qwen3.5-35B-A3B and the refusal-reduced HauhauCS-Aggressive fine-tune, we characterize one routed expert, Expert 114 at layer 14, as a linear readout of this register, and bound what it does. Across held-out, bottom-up, and cross-model tests we show that (1) its recovered router direction separates reflective-worldview-register generations from lexically matched controls with separated ranges (Cohen's d = 3.88); (2) a blind, prompt-independent auto-interpreter recovers the same register at AUC 0.94, broadening it beyond self-reference to abstract examination and philosophical-worldview language; (3) the detector is a readout with weak, conditional control: residual injection induces the register, gate down-bias leaves it intact, and the readout is stable across affirmative and skeptical interiority verdicts; and (4) the role is model-specific: index 114 is local to Qwen3.5-35B-A3B. Model-directed prompts served the discovery and dissociation stages; the coherent-window ladder measures target-directed vantage prompts over rock, river, tree, thermostat, cat, person, all-holding, and God, with a later AI-hidden-state follow-up near the low end of that ladder. We release the prompts, scripts, and provenance under the MIT license. This deposit is the named official paper-scoped artifact bundle. It includes a standalone PDF for Zenodo preview, the LaTeX source, the artifact zip, and the figure-backing data archive. The v1.2.5 update includes the AI-hidden-state follow-up rung to the target-directed prompt ladder and regenerates the ladder figure. Contact: Jeffrey W. Shorthill (jws299792@icloud.com). v1.3.0: adds the conversational-dynamics arm (convdyn/): two frozen preregistrations (grid v1.5, temperature battery v1.2, SHA-256 in-pack), frozen inputs (probe bank, orderings, paraphrase histories with authorship note), on-box drivers, analysis scripts, and scored results (phasic sharpening; composition-metering with the paraphrase manipulation establishing copy availability as causal). Paper updated with §7, related work, and limitations (v)-(vi).

Read Full Paperexternally

Ask AI

Helpful

Bookmark

View Full Paper