The Calculation of Errors by the Method of Least Squares

Present status of least squares' calculations. ---There are three possible stages in any least squares' calculation, involving respectively the evaluation of (1) the most probable values of certain quantities from a set of experimental data, (2) the reliability or probable error of each quantity so calculated, (3) the reliability or probable error of the probable errors so calculated. Stages (2) and (3) are not adequately treated in most texts, and are frequently omitted or misused, in actual work. The present article is concerned mainly with these two stages. Validity of the Gaussian error curve. ---All least squares' calculations of probable error assume that the residuals follow a Gaussian error curve. This curve is derived from a consideration only of accidental errors. Probable errors are, however, evaluated frequently in cases where constant or systematic errors are known to be present. Such a procedure, when used judiciously, is believed by the writer to be better than any alternative procedure, but the results are naturally less reliable than a strict reliance on theory would indicate. The statement is sometimes made that in practise one often gets more large residuals than are predicted by theory. This point is tested by 500 measurements of a spectral line, and the resulting Gaussian curve, plotted in Fig. 1, shows no indication of such a deviation. It is possible to account for an excess of large residuals (with the necessary accompaniment of a deficiency of small residuals), by assuming that the various observations used were not in fact of equal reliability. The formulas by which the probable error r may be calculated from the observed residuals, and the reliability of each such value, are briefly considered. Internal versus external consistency. ---Probable errors are calculated usually on the basis of internal consistency, and most texts discuss only this method. Scarborough, in a recent article, claims that there is no logical basis for the calculation from external consistency. This matter is considered in detail, and it is shown that the two methods must necessarily lead to the same result, except for statistical fluctuations, provided that only accidental errors are present. Formulas for the magnitude of the expected fluctuations are given. It is shown that a probable error based on internal consistency (r₈) is virtually a prediction and that the probable error based on external consistency (r₄) is the answer to this prediction. When the ratio r₄{r₈} exceeds unity by an amount much greater than is to be expected on the basis of statistical fluctuation, one has almost certain evidence of the presence of systematic errors. In such a case new arbitrary weights should be assigned. Then one has available only external consistency as a basis for the calculation of errors. The false deductions that result from a failure to note the above facts are illustrated by several examples from the literature, and a numerical problem illustrating all of these relations is presented in detail. Probable error of a function evaluated by least squares. ---The probable error of a function of directly observed quantities is given by the well-known law of "propagation of Error. " When, however, one has a function whose coefficients have been evaluated by least squares, the probable error of the function is scarcely mentioned in the literature, and the writer has never seen it calculated in practise. Explicit formulas for the error of a rational integral function of any degree are derived by the writer, from elementary considerations, and are found to agree with the very general formula already known. Practically all of the results presented here find frequent application in the article on probable values of e, h, etc. , which immediately follows.