Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information

This paper concerns normal approximations to the distribution of the maximum likelihood estimator in one-parameter families. The traditional variance approximation is 1/§, where θ is the maximum likelihood estimator and § is the expected total Fisher information. Many writers, including R. A. Fisher, have argued in favour of the variance estimate 1/I(x), where I(x) is the observed information, i.e. minus the second derivative of the log likelihood function at θ given data x. We give a frequentist justification for preferring 1/I(x) to 1/§. The former is shown to approximate the conditional variance of 8 given an appropriate ancillary statistic which to a first approximation is I(x). The theory may be seen to flow naturally from Fisher's pioneering papers on likelihood estimation. A large number of examples are used to supplement a small amount of theory. Our evidence indicates preference for the likelihood ratio method of obtaining confidence limits.