Investigating loss landscape curvature in neural networks through self-consistency terms shows critical weight implications for architectures.
We investigate the geometric effect of adding a self-consistency term of the form ‖f_θ(θ) - θ‖² to a standard task loss in neural networks. Such terms appear increasingly in meta-learning, world models, and reflective architectures, yet their effect on the loss landscape curvature remains poorly understood. We derive the exact Hessian of this augmented loss and show that it decomposes into a positive semidefinite component from linearization and an indefinite component arising from second-order nonlinearities. This indefinite component can induce a curvature bifurcation at a critical weight α_c, where the minimum eigenvalue of the total Hessian crosses zero. Using numerical experiments in high-dimensional settings (n=50-200) with realistic task Hessians and multiple random parameter points, we demonstrate that this phenomenon is robust and reproducible, yielding α_c = 1.85 ± 0.11 under our experimental conditions. The work resolves previously mysterious instabilities in reflective architectures (like the Godelian Collapse) and offers design guidelines for meta-learning and world models.
No takes yet. Share an insight, caveat, or question.
Moez Abdessattar (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: