Calibrating Confidence Coefficients

Abstract Two approaches for dealing with the problem of poor coverage probabilities of certain standard confidence intervals are proposed. The first is a recommendation that the actual coverage be estimated directly from the data and its value reported in addition to the nominal level. This is achieved through a combination of computer simulation and density estimation. The asymptotic validity of the procedure is proved for a number of common situations. A classical example is the nonparametric estimation of the variance of a population using the normal-theory interval. Here it is shown that the estimated coverage probability consistently estimates the true coverage probability if the population distribution possesses a finite sixth moment. The second approach is more traditional. It is a procedure for modifying an interval to yield improved coverage properties. Given a confidence interval, its estimated coverage probability obtained in the first approach is used to alter the nominal level of the interval. The interval with this modified nominal level is called a calibrated interval. In the case that the given interval is the normal-theory interval for the estimation of variance, the calibrated interval is proved to be asymptotically robust as long as sixth moments exist. As another application, the method is used to modify a bootstrap interval procedure for variance estimation. This leads to the derivation of a new bootstrap interval.