Posterior Concentration Rates for Bayesian Penalized Splines

Despite their widespread use in practice, the asymptotic properties of Bayesian penalized splines have not been investigated so far. We close this gap and study posterior concentration rates for Bayesian penalized splines in a Gaussian nonparametric regression model. A key feature of the approach is the hyperprior on the smoothing variance, which allows for adaptive smoothing in practice but complicates the theoretical analysis considerably. Our main tool for the derivation of posterior concentration rates with a general hyperprior on the smoothing variance is a novel spline estimator that projects the observations onto the first basis functions of a Demmler-Reinsch basis. Our results show that posterior concentration at near optimal rate can be achieved if the hyperprior on the smoothing variance strikes a fine balance between oversmoothing and undersmoothing. Another interesting finding is that the order of the roughness penalty must exactly match the regularity of the unknown regression function in order to achieve posterior concentration at near optimal rate. Overall, our results are the first posterior concentration results for Bayesian penalized splines and can be generalized in many directions.