Answers are grounded in this paper and cite their sources.
We present a constancy rate principle governing language generation. We show that this principle implies that local measures of entropy (ignoring context) should increase with the sentence number. We demonstrate that this is indeed the case by measuring entropy in three different ways. We also show that this effect has both lexical (which words are used) and non-lexical (how the words are used) causes.
No takes yet. Share an insight, caveat, or question.
Genzel et al. (2001) studied this question.