Superposition of Safety-Critical Computations in LLM MLP Layers: A Grassmannian Subspace Analysis

Puntos clave

• The study investigates the encoding of safety-critical computations within MLP layers of transformer models.
• Analyzed MLP layers 21-29 in transformer models like Llama 3.1 8B and Mistral 7B-Instruct.
• Employed causal ablation and Grassmann distance metrics for evaluation.
• Determined the distinctness between refusal and hallucination subspaces using singular vectors.
• Establishes a high area under the curve (AUC=0.975) for refusal subspace accuracy.
• Identifies a significant Grassmann distance (0.94) between refusal and hallucination subspaces.
• Replicates findings across different transformer architectures, indicating a common design feature.