Answers are grounded in this paper and cite their sources.
Linear probes are widely used in LLM interpretability, but does high accuracy mean the probe found something causally important? We test this by training probes on refusal behavior in two models, Phi-3 (3.8B) and Mistral-7B, and then using activation interventions to check whether the probe directions are actually necessary. The answer depends on the model. In Phi-3, refusal lives in a low-rank subspace: removing it makes the behavior collapse. In Mistral-7B, refusal is distributed across many redundant directions: adding them steers behavior, but removing them does nothing. High probe accuracy does not tell you which regime you are in.
No takes yet. Share an insight, caveat, or question.
MS et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: