Residuals and Their Variance Patterns

One of the most important aspects of the analysis of data by regression methods is the examination of residuals. This implies the careful inspection of the differences, ei = yi – ŷ i , i = 1,2, …, n, between the observed values yi , and the corresponding values yi which are predicted by the fitted model at the n observation sites. There are many ways of looking at residuals; for basic references see Anscombe and Tukey (1963) and Anscombe (1961, 1967), or see the expository accounts by Draper and Smith (1966, Chapter 3) and Wooding (1969). Important basic techniques are the inspection of the magnitude of individual residuals to detect the presence of outliers and the plotting of the residuals against their corresponding fitted values, or against the corresponding values of the independent variables, or against the corresponding values of “new” variables, and (in all cases) observing the pattern thus formed. An “ideal” pattern for most plots, that implies no denial of the regression assumptions, occurs when the residuals give a “horizontal band” effect. This isalwaystrue for the Fisherian fixed effect factorial design models. In fitting models with continous variables, it is usually true within the practical limitations of most plots, but is not precisely true theoretically because the residuals do not all have the same variance. The purpose of this note is two-fold: (i) to point out that there is likely to be at least a slight pattern of changing magnitude of the residuals in such plots even when the assumptions are met, (ii) to suggest that the constancy of the variances of the residuals be checked before discarding an outlier or assuming a residuals plot has demonstrated model failure. Since most regression calculations are made on computers these days, the use of standardized residuals in such plots is a simple matter. Inspection of the residual variances may also point out an inherent inability of the data to test the adequacy of the model.