Comparison of Smoothing Parameterizations in Bivariate Kernel Density Estimation

Abstract The basic kernel density estimator in one dimension has a single smoothing parameter, usually referred to as the bandwidth. For higher dimensions, however, there are several options for smoothing parameterization of the kernel estimator. For the bivariate case, there can be between one and three independent smoothing parameters in the estimator, which leads to a flexibility versus complexity trade-off when using this estimator in practice. In this article the performances of the different possible smoothing parameterizations are compared, using both the asymptotic and exact mean integrated squared error. Our results show that it is important to have independent smoothing parameters for each of the coordinate directions. Although this is enough for many situations, for densities with high amounts of curvature in directions different to those of the coordinate axes, substantial gains can be made by allowing the kernel mass to have arbitrary orientations. The “sphering” approaches to choosing this orientation are shown to be detrimental in general, however.