Self-Correcting Neural Networks: Provable Robustness through Gradient Surgery

Standard neural networks learn through a single global feedback signal, backpropagation, and possess no intrinsic mechanism to detect or repair node-level damage during train- ing. When training conditions deteriorate due to pathological label noise, catastrophic weight perturbation, or vanishing gradients, this single signal fails to distinguish healthy computation from pathological memorization, causing the network to silently collapse. We propose three-signal training: a unified architecture combining global backpropa- gation, per-node local loss via a shared bottleneck readout, and loss-gated autonomous self-correction, where PCGrad gradient surgery dynamically resolves conflicts between these signals. We provide an Input-to-State Stability (ISS) convergence analysis—the first for any self-correcting architecture—proving that our Noether-inspired conservation constraint bounds the trajectory to a γ(ε) neighborhood. Our “GlassBox” observability enables per-node causal traceability: each weight correction is tagged with the specific pathology that triggered it, the node affected, and the diagnostic evidence. Across 309 comprehensive experimental runs under equal-compute conditions, ORMAS demon- strates three capabilities: (1) the first autonomous structural self-repair mid-training— recovering from catastrophic (σ= 1.0) weight damage to 75.4% accuracy where Mixup permanently collapses to 33.3%; (2) provably bounded late-stage convergence stability, reducing parameter variance to 0.19 (a 4.5×reduction over standard backpropagation); and (3) leading accuracy among parameter-matched single-network architectures under 40% symmetric label noise (78.1%), outperforming standard regularizers by over 28 percentage points without semi-supervised ensembling. Under equal 200-epoch training, baselines decay 3–4×faster than ORMAS, confirming that the three-signal system actively resists noise memorization rather than merely delaying it.