Answers are grounded in this paper and cite their sources.
The language model probabilities are estimated by an empirical Bayes approach in which a prior distribution for the unknown probabilities is itself estimated through a novel choice of data. The predictive power of the model thus fitted is compared by means of its experimental perplexity [1] to the model as fitted by the Jelinek-Mercer deleted estimator and as fitted by the Turing-Good formulas for probabilities of unseen or rarely seen events.
No takes yet. Share an insight, caveat, or question.
Arthur Nádas (1984) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: