Divergence Estimation for Multidimensional Densities Via k-Nearest-Neighbor Distances

A new universal estimator of divergence is presented for multidimensional continuous densities based on k-nearest-neighbor (k -NN) distances. Assuming independent and identically distributed (i. i. d. ) samples, the new estimator is proved to be asymptotically unbiased and mean-square consistent. In experiments with high-dimensional data, the k-NN approach generally exhibits faster convergence than previous algorithms. It is also shown that the speed of convergence of the k-NN method can be further improved by an adaptive choice of k.