An Optimal Property of Principal Components in the Context of Restricted Least Squares

A new optimal property for principal components regression is presented. In particular, it is shown that the trace of the covariance matrix for estimators obtained by deleting principal components associated with the smallest eigenvalues is at least as small as that for any other least-squares estimator with an equal or smaller number of linear restrictions. This property is useful in suggesting data transformations and determining the maximum variance reduction obtainable from the introduction of linear restrictions on the parameter space.