Currently, there is little agreement as to how Natural Language Generation (NLG) systems should be evaluated, with a particularly high degree of variation in the way that human evaluation is carried out. This paper provides an overview of how human evaluation is currently conducted, and presents a set of best practices, grounded in the literature. With this paper, we hope to contribute to the quality and consistency of human evaluations in NLG.
No takes yet. Share an insight, caveat, or question.
Lee et al. (2019) studied this question.
Synapse has enriched 4 closely related papers on similar clinical questions. Consider them for comparative context: