On the Multiple Descent of Minimum-Norm Interpolants and Restricted Lower Isometry of Kernels

We study the risk of minimum-norm interpolants of data in Reproducing Kernel Spaces. Our upper bounds on the risk are of a multiple-descent shape the various scalings of d = n^\, \\ (0, 1), for the input d and sample size n. Empirical evidence supports our finding that-norm interpolants in RKHS can exhibit this unusual non-monotonicity in size; furthermore, locations of the peaks in our experiments match our predictions. Since gradient flow on appropriately initialized wide networks converges to a minimum-norm interpolant with respect to a kernel, our analysis also yields novel estimation and generalization for these over-parametrized models. At the heart of our analysis is a study of spectral properties of the random matrix restricted to a filtration of eigen-spaces of the population operator, and may be of independent interest.