Assessing the Validity of Lexical Diversity Indices Using Direct Judgements

Indices of lexical diversity have been used to estimate the size of a writer’s vocabulary and/or a writer’s lexical proficiency for some time. One issue with many commonly used indices of lexical diversity (e.g., TTR and index) is that they vary as a function of text length. Accordingly, much research has been devoted to the development of indices that are text length independent. However, very little research has investigated the degree to which indices of lexical diversity are reflective of human ratings of diversity itself. In this study, the relationship between indices related to three dimensions of lexical diversity (abundance, variety, and volume) and human ratings of lexical diversity are explored in a corpus of L1 and L2 argumentative essays. The results indicated that abundance was the strongest predictor of lexical diversity ratings, followed by volume and variety. Multivariate models indicated that 74% of the variance in lexical diversity ratings could be explained by abundance and variety, and that speaker status had a negligible effect. Implications for research, writing instruction, and writers are discussed.