What question did this study set out to answer?

March 10, 2026Open Access

Refusal Geometry in LLMs: A Universal Property of Late MLP Layers Evidence from Grassmannian Subspace Analysis Across Three Model Families

Key Points

This research aims to explore the geometric structure of refusal behavior in large language models across different architectures.
Analyzed the geometric structure of refusal behavior using Grassmannian subspace analysis.
Evaluated three model families: Llama 3.1 8B, Mistral 7B, and Gemma 2 9B.
Conducted experiments on a test set of 30 pairs derived from 100 total samples.
Refusal geometry peaks in the last 10% of transformer layers for all model families.
Llama achieved an out-of-distribution area under the curve (AUC) of 0.969.
MLP layers 21-29 specifically construct the refusal subspace with an ablation delta of -0.396.
Refusal and hallucination subspaces were found to be nearly orthogonal with a Grassmann distance of 0.944.

Abstract

We investigate the geometric structure of refusal behavior in large language models using Grassmannian subspace analysis across three independently trained model families: Llama 3.1 8B (Meta), Mistral 7B (Mistral AI), and Gemma 2 9B (Google). Our key finding is that refusal geometry consistently peaks in the last 10% of transformer layers across all three architectures. On a held-out test set of 30 pairs from 100 total, Llama achieves OOD AUC = 0.969. MLP layers 21-29 causally construct refusal subspace (ablation delta = -0.396), and refusal and hallucination subspaces are nearly orthogonal (Grassmann distance = 0.944), consistent with the superposition hypothesis. Research conducted with assistance from Claude (Anthropic).

Read Full Paperexternally

اسأل الذكاء الاصطناعي

Bookmark

View Full Paper