Notes on Bias in Estimation

One of the commonest problems in statistics is, given a series of observations Xj, xit. . . , xn, to find a function of these, tn (xltxit. . . , xn), which should provide an estimate of an unknown parameter 0. The desirable properties of estimation procedures have been discussed fully elsewhere. They are: (a) That the estimator should be efficient according to some definition of efficiency pre-viously arranged. Most commonly, the reciprocal of the variance of the estimates is taken as a measure of its efficiency, as this is most useful where central limit theory may be relevant. (6) That the estimator should utilize all the information contained in the observations, x1, xi,. . . , xn concerning the parameter 6. This is not always possible, but, if such an estimator exists, it is called sufficient. (c) That the estimator should be consistent, i. e. tn converges in some probabilistic sense to 6, usually lim tn-* • 6. n-»-a (d) That the estimator should be unbiased, i. e. E (tn) = 6. The method of maximum likelihood is popular in that it satisfies properties (a) to (c), whence, by evaluating E (tn), an unbiased statistic may be derived. That such evaluation is necessary is obvious when it is remembered that i/r (tn) is, by the same theory, the estimator of ft (6), and, since in general Efr (tn) ^ iJrE (tn), it will be the exception rather than the rule for a maximum-likelihood estimator to be unbiased. Provided the exceptions may be simply evaluated no real difficulty arises. However, often the complexity of the evaluation presents a major drawback and some simple approach is then desirable. 2. If the observations are taken in random order, the estimator tn may often be written where klt kit. . . , km are unbiased estimates of the cumulants K1, K2I. . . , m. Then, provided that (a) m is independent of n, (6) the function tn is capable of Taylorian expansion, (c) all of the cumulants are finite, (d) tn is consistent, i. e. 6 = lim (n (A;1. . . km), n-no it follows that tn-e = Since the moments of the estimators, kit are power series in 1/n, it follows that E (tn — 8), i. e. the bias in tn, is also expressible as a power series in 1/n.