A class of new KNN methods for low sample problems

The K-nearest-neighbor (KNN) estimates proposed by D.O. Loftsgaarden and C.P. Quesenbery (1965) give unbiased and consistent estimates of the probability density function of a random variable from N observations of that random variable when K, the number of nearest neighbors considered, and N, the total number of observations available, tend to infinity such that K/N to 0. A class of new KNN estimates is proposed as weighted averages of K KNN estimates, and it is shown that in small sample problems they give closer estimates to the true probability density than the traditional KNN estimates. On the basis of some experimental results, the KNN rules based on these estimates are shown to be suitable for small sample classification problems.>