Perturbation Robustness Profiles Reveal Heterogeneous Capability Degradation and an Evaluation-Modality Blind Spot in Instruction-Tuned Language Models

We study whether instruction-tuned language-model behaviors have the same robustness profile under controlled weight perturbation. We inject per-tensor Gaussian noise into Llama-3.1-8B-Instruct and evaluate four capability families. Capabilities degrade heterogeneously (IFEval retains 82.2% at σ=0.2, GSM8K retains 45.1%; p=0.033). Safety refusal behavior shows qualitatively higher seed-level variance (CV=67.6% vs <13% for capabilities). A component-level sweep initially appeared to show capability-type dissociation; a modality control shows the stronger result is evaluation-modality dissociation: MMLU log-likelihood scoring retains 47.4% at σ=2.0 on layer_29_attention while greedy-generation collapses to 4.4% with 93.1% extraction failure. Log-likelihood evaluation cannot certify deployment-relevant generation capability after weight modification. Preprint v2.1 — companion code at https://github.com/mohitdak24/perturbation-robustness-profiles