What question did this study set out to answer?

This research aims to investigate whether activation sparsity improves neuron monosemanticity in large language models.

June 1, 2026Open Access

Activation Sparsity Does Not Independently Improve Neuron Monosemanticity: An Apparent Effect Explained Away by Network Depth

Key Points

This research aims to investigate whether activation sparsity improves neuron monosemanticity in large language models.
Compared a sparsified ReLU model (MiniCPM-S-1B) with a dense SiLU counterpart on neuron monosemanticity.
Measured monosemanticity using Jensen–Shannon scores and variance-based η² scores across layers.
Regressed out network depth to assess correlation between per-layer sparsity and monosemanticity gains.
The sparse model appeared more monosemantic in deeper layers, but this effect was fully explained by network depth.
After adjusting for depth, no detectable correlation was found between sparsity and monosemanticity gain (residual r ≈ 0.12).
Findings challenge previous claims linking sparsity and monosemanticity in certain models, suggesting specificity of sparsity forms.

Abstract

Activation sparsity is widely used to accelerate large language model inference, and is often claimed to make neurons more interpretable, or monosemantic. We test this claim directly, comparing an efficiency-driven ReLU-sparsified model (MiniCPM-S-1B) against its dense SiLU sibling on per-layer neuron monosemanticity, measured by both a Jensen–Shannon concept-separability score and a variance-based η² score. While the sparse model appears more monosemantic in deeper layers, we show this advantage is fully explained by network depth: after regressing out depth, per-layer sparsity has no detectable correlation with the monosemanticity gain (residual r ≈ 0.12). This null result is robust across two orthogonal concept dimensions (topic and sentiment) and two methodologically distinct metrics. Our finding stands in tension with reports of sparsity-induced monosemanticity in mixture-of-experts models, suggesting that any link between sparsity and monosemanticity may depend on the specific form of sparsity rather than holding universally.

Read Full Paperexternally

Ask AI

Helpful

Bookmark

View Full Paper