Answers are grounded in this paper and cite their sources.
The evaluation of conversational dialog systems has remained a controversial topic, as it is challenging to quantitatively assess how well a conversation agent performs, or how much better one is compared to another. Furthermore, one of the hurdles which remains elusive in this quandary is the definition of naturalness, as demonstrated by how well a dialog system can maintain a natural conversation flow devoid of perceived awkwardness. As a step towards defining the dimensions of effectiveness and naturalness in a dialog system, this paper identifies existing evaluation practices which are then expanded to develop a more suitable assessment vehicle. This method is then applied to the LifeLike virtual avatar project.
No takes yet. Share an insight, caveat, or question.
Hung et al. (2009) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: