Beyond Gradient Fidelity in Gaussian Precision Learning

Matching the gradient does not determine the value of a finite update in Gaussian precision learning. We show how componentwise inverse information preserves gains that parameter-wise separation loses, and quantify how local approximation accuracy can fail to preserve those gains. For positive affine precision models, retaining prior–posterior cancellation yields a reciprocal minorizer that preserves shared-update geometry and supports boundary activations lost by separation. On two natural instances, certified component-model gains exceed the separated models' global optima by at least 12.37 and 42.70 nats. The approximation limit is sharper still: reciprocal models can agree in old value, gradient and Hessian, with a factor-two remainder envelope, yet select actions retaining only an inverse-linear fraction of available gain. Bounding feasible edge ratios restores a matching-order guarantee, establishing Θ(T⁻¹) worst-case dependence on action scale for the factor-two class. In a Gaussian realization, the retained fraction of actual likelihood gain also vanishes. Original-matrix certificates carry the analysis into executed updates: complete rounded learning paths attain gains of 158.63 and 134.13 nats. These results identify finite-action value as a criterion for choosing what inverse information a learning algorithm retains.