Answers are grounded in this paper and cite their sources.
Suppose P is an arbitrary discrete distribution on acountable alphabet š³. Given an i.i.d. sample ( X 1 ,ā¦, X n ) drawnfrom P , we consider the problem of estimating the entropy H ( P ) or some other functional F = F ( P ) of the unknown distribution P . We show that, for additive functionals satisfying mild conditions (including the cases of the mean, the entropy, and mutual information), the plugāin estimates of F are universally consistent. We also prove that, without further assumptions, no rateāofāconvergence results can be obtained for any sequence of estimators. In the case of entropy estimation, under a variety of different assumptions, we get rateāofāconvergence results for the plugāin estimate and for a nonparametric estimator based on matchālengths. The behavior of the variance and the expected error of the plugāin estimate is shown to be in sharp contrast to the finiteāalphabet case. A number of other important examples of functionals are also treated in some detail.
No takes yet. Share an insight, caveat, or question.
Antos et al. (2001) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: