Principal Components, Sufficient Dimension Reduction, and Envelopes

We review probabilistic principal components, principal fitted components, sufficient dimension reduction, and envelopes, arguing that at their core they are all based on variations of the conditional independence argument that Fisher used to develop his fundamental concept of sufficiency. We emphasize the foundations of the methods. Methodological details, derivations, and examples are included when they convey the flavor and implications of basic concepts. In addition to the main topics, this review covers extensions of probabilistic principal components, the central subspace and central mean subspace, sliced inverse regression, sliced average variance estimation, dimension reduction for covariance matrices, and response and predictor envelopes.