Exploiting the connection between PLS, Lanczos methods and conjugate gradients: alternative proofs of some properties of PLS

The connection between partial least squares regression (PLS) and Lanczos methods for approximating the extremal eigenvalues of a symmetric matrix has long been known. Less well known, however, is that PLS is in fact identical to a common implementation of the conjugate gradient algorithm for solving the normal equations. In this paper we outline the connections between, on the one hand, PLS and, on the other, Lanczos methods and conjugate gradients. In addition to shedding more light on PLS, these connections allow us to provide alternative and somewhat simpler proofs of two of its well‐known properties: first, that it yields a shrinkage estimator; and second, that it ‘fits better’ than principal component regression.