Asymptotic Theory of Efficient Nonparametric Estimation of the Ratio of Two Densities

The problem of nonparametric estimation of the ratio of two densities is a classical topic in discriminant analysis, machine learning and statistical classification. The aim of the paper is to develop the theory of efficient nonparametric estimation of the ratio under the mean integrated squared error (MISE) criterion. For the first time in the literature, a sharp lower bound for minimax MISE is developed for the oracle who knows data, an underlying Sobolev functional class of the ratios, and the reference (denominator) density. Then a data driven estimator is proposed that matches performance of the oracle. It is shown that the problem of ratio estimation may be dramatically more complicated than estimation of a density, and its complexity is captured by the coefficient of difficulty which is a special functional of the two densities defining the ratio.