This work develops approximations of Hilbert-Schmidt operators and Bayesian posteriors on Hilbert spaces. We construct generalised rank-constrained approximations of Hilbert-Schmidt operators by extending the Eckart-Young theorem. Furthermore, we construct low-rank approximations of posteriors for linear Gaussian inverse problems by solving a family of measure approximation problems. Linear Gaussian inverse problems are linear inverse problems with Gaussian priors and Gaussian observation noise. Both approximations are used in methods that aim at reducing complexity, for example in large-scale inverse problems and model reduction. By analysing their properties on separable Hilbert spaces, we provide an infinite-dimensional description of these approximation methods. In this way, we obtain a discretisation-independent perspective and can uncover intrinsic low-dimensionality. Generalised rank-constrained approximations are important in the field of model reduction and are derived via an extension of the Eckart-Young theorem. Such approximations are obtained by solving a minimisation problem in the Hilbert-Schmidt norm, which reduces to the Frobenius norm in finite dimensions. We show that existence and continuity of such minimisers may not hold in infinite dimensions. Therefore, we establish necessary and sufficient conditions for existence, uniqueness, and continuity, and give an explicit expression of the minimisers. We also construct continuous approximations of discontinuous minimisers, and correct a minimal norm property that minimisers were believed to satisfy in the finite-dimensional setting. To further motivate the use and generality of our results, we apply them to analytically solve a problem in reduced-rank linear operator learning, which is related to problems in signal processing and reduced-rank regression. Low-rank approximations of linear Gaussian inverse problems enable the resolution of linear Gaussian inverse problems that are large-scale, typically originating from fine discretisations of the equations governing the inverse problem. In such discretised, finite-dimensional settings, low-rank approximations of the posterior mean and the posterior covariance that satisfy optimality properties have been determined. In infinite dimensions, low-rank approximations may yield Gaussian probability measures that are mutually singular to the exact Gaussian posterior. We take this into account and formulate approximation problems for several divergences on probability measures, which are finite only for non-singular approximations. This uncovers a fundamental connection between the low-rank approximation procedure and the theory of equivalent Gaussian measures. Furthermore, we characterise the low-rank approximations of the posterior mean, posterior covariance, and posterior precision, that yield non-singular posterior approximations, and identify the optimal ones by minimising the divergences. A joint approximation problem for the mean and covariance is formulated in the reverse Kullback-Leibler divergence, averaged over the data distribution. By solving this problem, we establish that low-rank methods can optimally approximate the entire Gaussian posterior distribution. This perspective is further explored by determining a projected inverse problem for which the posterior corresponds to the optimal low-rank approximation. The projector identifies the directions in the parameter space that are informed the most by the data, relative to the prior, showing that the prior-to-posterior update on the infinite-dimensional space typically satisfies low-dimensional behaviour. Our discretisation-independent analyses provide fundamental insights into the theory of generalised rank-constrained approximations of Hilbert-Schmidt operators and low-rank approximations of linear Gaussian inverse problems. This enables the establishment of dimension-independent results for specific linear or linearised inverse problems and applications in model reduction, and has potential applications in related areas of inverse problems, statistics, probability, and dimension reduction.
Giuseppe Carere (Thu,) studied this question.