The language model probabilities are estimated by an empirical Bayes approach in which a prior distribution for the unknown probabilities is itself estimated through a novel choice of data. The predictive power of the model thus fitted is compared by means of its experimental perplexity [1] to the model as fitted by the Jelinek-Mercer deleted estimator and as fitted by the Turing-Good formulas for probabilities of unseen or rarely seen events.
No takes yet. Share an insight, caveat, or question.
Arthur Nádas (1984) studied this question.
Synapse has enriched 2 closely related papers on similar clinical questions. Consider them for comparative context: