Investigates noise-resolved effective rank in flow-matching models, indicating generalization capacity.
We introduce a single, scale-resolved diagnostic for how much of the data a diffusion or flow-matching model has learned: the noise-resolved effective rank of the denoiser, D_eff(t,x) = tr(∂_x D_t(x)), the number of data directions the model resolves at noise scale σ_t. It is computed from any trained model by a cap-free stochastic Lanczos quadrature on the symmetrized denoiser Jacobian, with negativity-ratio and nonnormality diagnostics that indicate when a learned, non-symmetric Jacobian admits a meaningful rank reading. The quantity is not ad hoc. In a Gaussian–linear core, the averaged effective rank is exactly the ridge degrees of freedom Σ_j λ_j/(λ_j+σ_t²), whose sample-rank saturation marks the population-to-sample transition; via the second-order Tweedie identity ∂_x D_t = σ_t^(−2) Cov[x_0 | x_t] it is a posterior-covariance trace for general data, symmetric and positive semidefinite for the Bayes-optimal denoiser but not for a generic learned network; and it renders the diffusion "intrinsic dimension" explicitly scale-dependent — the noise-resolved rank #{λ_j > σ_t²} — reconciling local (FLIPD-type) and global (Pope-type) estimates. We give conditional 2-Wasserstein propagation support (a synchronous-coupling contraction–discount relation for the probability-flow ODE, with the eigenframe-rotation gap closed under explicit reach and curvature assumptions) and, with explicit constants, an intrinsic-dimension score-estimation rate for the empirical-Bayes denoiser on general C² manifolds, where a Tweedie-to-denoiser argument cancels the ambient dimension. Empirically, the effective rank is strongly scale-dependent on a pretrained CIFAR-10 diffusion U-Net and on a production SiT-XL/2 transformer; on a controlled manifold it tracks the generalization gap; and it yields a generation-free, label-free probe that separates memorizing from generalizing flow-matching models on real images. We do not claim a neural-network training guarantee: for learned networks the score-estimation rate reduces to a realization residual relative to the empirical-Bayes denoiser, whose optimization part we measure rather than bound.
No takes yet. Share an insight, caveat, or question.
Munsik Kim (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: