Ridge regression

Ridge regression is a popular parameter estimation method used to address the collinearity problem frequently arising in multiple linear regression. The formulation of the ridge methodology is reviewed and properties of the ridge estimates capsulated. In particular, four rationales leading to a regression estimator of the ridge form are summarized. Algebraic properties of the ridge regression coefficients are given, which elucidate the behavior of a ridge trace for small values of the ridge parameter (i.e., close to the least squares solution) and for large values of the ridge parameter. Further properties involving coefficient sign changes and rates‐of‐change, as functions of the ridge parameter, are given for specific correlation structures among the independent variables. These results help relate the visual behavior of a ridge trace to the underlying structure of the data.