We apply structural fragility analysis a domain-agnostic graph theoretic framework originally validated on molecular signalling networksto the internal topology of transformer attention mechanisms. We extract co-activation graphs from BERT-base (144 nodes, 12 lay ers × 12 heads) and compute per-edge triangle participation counts tri(e) and the Fragility Index (FI), which quanti es the proportion of connections with zero local redundancy. At a correlation thresh old of 0.8, BERT exhibits FI = 0.97%, con rming massive structuralover-redundancy that quantitatively explains the known tolerance of transformers to aggressive head pruning.Progressive layer-wise pruning (110 heads per layer, 3 trials, SST-2 validation) reveals three functional regimes: (i) immune lay ers (L1, L7, L10) with near-zero degradation even at 83% head removal, (ii) critical layers (L2, L3, L5, L6, L8, L9) exhibiting smooth monotonic decay, and (iii) bu er layers (L0, L4, L11) with delayed onset. Crucially, grouped multi-layer pruning reveals a structural phase transition in pruning sensitivity: at low perturbation budgets, topologically fragile layers degrade faster (∆ = −0.58% vs −0.45%), but at high budgets, topologically redundant layers collapse disproportionately (∆ = −4.18% vs −1.53% at 8 heads/layer). Normalised per head pruned, damage from robust heads doubles (0.075% vs 0.038%), with superlinear ampli cation characteristic of cooperative failure above a percolation threshold.This reverses the intuitive interpretation of redundancy: highly triangulated structures are not structurally safe but mark regions of concentrated functional dependency. The analysis is purely structural, requiring no gradient computation or ne-tuning, making it computationally efficient and inherently parallelisable for large-scale transformer architectures
David Martin Venti (Tue,) studied this question.