Adaptive Bayesian Regression on Data with Low Intrinsic Dimensionality

We study how the posterior contraction rate under a Gaussian process (GP) prior depends on the intrinsic dimension of the predictors and smoothness of the regression function. An open question is whether a generic GP prior that does not incorporate knowledge of the intrinsic lower-dimensional structure of the predictors can attain an adaptive rate for a broad class of such structures. We show that this is indeed the case, establishing conditions under which the posterior contraction rates become adaptive to the intrinsic dimension in terms of the covering number of the data domain X (the Minkowski dimension), and prove the optimal posterior contraction rate O (n^-s/ (2s +) ), up to a logarithmic factor, assuming an approximation order s of the reproducing kernel Hilbert space (RKHS) on X. When X is a -dimensional compact smooth manifold, we study RKHS approximations to intrinsically defined s-order H\"older functions on the manifold for any positive s by a novel analysis of kernel approximations on manifolds, leading to the optimal adaptive posterior contraction rate. We propose an empirical Bayes prior on the kernel bandwidth using kernel affinity and k-nearest neighbor statistics, eliminating the need for prior knowledge of the intrinsic dimension. The efficiency of the proposed Bayesian regression approach is demonstrated on various numerical experiments.