Register, Not Risk: Decomposing Harm Detection in Language-Model Representations

Linear probes on hidden states of large language models achieve about 0.90 balanced accuracy in separating harmful from benign prompts, but this does not necessarily mean the models linearly encode a harm concept; instead, much of the probe signal arises from a procedural-register confound, where harmful prompts are typically instructional and benign prompts are often question-answering, allowing even a simple TF-IDF classifier to match or outperform geometric probes. Across Mistral-7B, Qwen2.5-7B, Qwen2.5-1.5B, and sparse autoencoder analysis on Gemma-2-2B, erasing the register direction removes roughly 55–63% of the above-chance signal, while the remaining register-independent signal is genuine only within the same dataset and fails to transfer across different harm datasets or harm types. A register-controlled evaluation shows only a modest residual accuracy, and on implicit harm datasets such as ToxiGen, all methods perform poorly. These findings suggest that probe detectability should not be treated as evidence of concept encoding and that safety evaluations should include cross-source transfer tests, register-controlled lexical baselines, and interpretable features, which are necessary but not sufficient to demonstrate true concept representation.