Some Comments on Bivariate Regression

We discuss the relationship between the multiple correlation coefficient and the correlations among the variables in a bivariate regression. We point out a common error concerning the variation explained by regression variables. A geometric interpretation is given and examples to illustrate the error are provided. example, the proportion of variation in y explained by x1 andx2 together can be greater than the sum of the proportions explained by x1 alone and x2 alone. It would appear that this situation may be misunderstood; in fact, two recent textbooks on regression actually state that this cannot arise. We present results, diagrams and examples which show that it does arise and that it may be quite common. We will call such an occurrence enhancement. The fitted regression coefficients can also exhibit bizarre behaviour. If we compare the fitted coefficient of, say, x1 in the regression of y on x1 with the fitted coefficient of x1 in the bivariate regression then these coefficients may differ by orders of magnitude or be of opposite sign. Such behaviour is sometimes called suppression. We show that there is a close connection between enhancement and suppression. We give examples taken both from bivariate and multiple regression to illustrate these phenomena.