Probing Algebraic Structure in Neural Networks via Symmetric Cycle Closure

## Abstract Mechanistic interpretability of grokking [Power et al. 2022] often relies on label accuracy or geometric proxies to argue that a network has learned algebraic structure. We argue these signals are not enough and introduce **Symmetric Cycle Closure Count (Sym-CCC)**, a functional probe based on group axioms. It tests bi-directional cancellation: a network respects group G if both Net(Net(a,b), b⁻¹) = a and Net(a⁻¹, Net(a,b)) = b hold over G × G. We derive the random baseline analytically (1/|G|², Lemma 3.1) and confirm it on untrained networks. We test this probe on two non-abelian groups of order 24 — the dihedral group D₁₂ and the symmetric group S₄ — across multiple seeds, and for S₄, across independently-randomized train/val splits with a complete, pre-registered $n=5$ seed set. Four findings: 1. **Weight decay separates algebraic generalization from algebraic overfitting.** On D₁₂ (5 seeds), removing weight decay yields 100% train accuracy with Train Sym-CCC = 53.9% ± 3.1% but Val Sym-CCC = 5.7% ± 1.8%. With weight decay (0.1), both rise to 79–92%. On S₄ (5 seeds, closed-domain evaluation with a fixed, task-derived denominator), the separation is sharpest: **exact-zero, zero-variance Val Closed Strict Sym-CCC** without weight decay across all 5 seeds, versus 54.7% ± 31.2% with weight decay, at a pre-registered 4/5 grokking rate. 2. **Coordinate alignment speeds up but does not decide structure learning.** On D₁₂, random index permutation reaches Val Sym-CCC = 72.6% ± 9.2% versus 79.4% ± 10.4% under contiguous indexing. 3. **A late-phase Float32 dip is linked to precision noise, not Softmax overflow**, on D₁₂, supported by two direct interventions (AdamW ε, Float64). A matching dip observed on S₄ is reported as an unconfirmed, observational parallel. 4. **The core separation replicates on a structurally distinct group** under an independent architecture and evaluation protocol. We report all statistics with explicit caveats, including small sample sizes at the seed level ($n=5$) and candidate-pair level (16–26 pairs per split for S₄), an explicit statement of our SD convention, a formal power argument for the exact-zero result, a pre-registration statement for seed selection, and a rare (but non-zero) exception to metric-equality that we flag rather than over-explain. **Keywords:** grokking, mechanistic interpretability, group theory, weight decay, generalization, functional probes, numerical precision, closed-domain verification, multi-seed statistics, pre-registration