Refusal Geometry in LLMs: A Universal Property of Late MLP Layers Evidence from Grassmannian Subspace Analysis Across Three Model Families

Key Points

• This research aims to explore the geometric structure of refusal behavior in large language models across different architectures.
• Analyzed the geometric structure of refusal behavior using Grassmannian subspace analysis.
• Evaluated three model families: Llama 3.1 8B, Mistral 7B, and Gemma 2 9B.
• Conducted experiments on a test set of 30 pairs derived from 100 total samples.
• Refusal geometry peaks in the last 10% of transformer layers for all model families.
• Llama achieved an out-of-distribution area under the curve (AUC) of 0.969.
• MLP layers 21-29 specifically construct the refusal subspace with an ablation delta of -0.396.
• Refusal and hallucination subspaces were found to be nearly orthogonal with a Grassmann distance of 0.944.