A general approach to bias reduction and positive definiteness in covariance estimation

Different analytical and numerical methods have been proposed to reduce the bias of estimators of – possibly vector-valued – parameters that index statistical models. These methods aim to mitigate systematic estimation errors, particularly in small samples. The most common bias-correction approach consists of subtracting an estimated bias from the original estimator, which generally leads to a reduction in bias. However, when this strategy is applied to the estimation of a covariance matrix, the corrected estimator may lose positive definiteness. In this paper, we introduce a general bias-reduction framework specifically designed for covariance matrix estimators that preserves positive definiteness. We then apply our main result to the estimation of the covariance matrix of the ordinary least squares estimators in the linear regression model with heteroskedastic errors.