Cluster-based user simulations for learning dialogue strategies

Abstract Good dialogue strate gies in spok en dialogue systems help to en-sure and maintain mutual understanding and thus play a crucialrole in rob ust con versational interaction. W e focus on clariÞca-tion strate gies and build user simulations which are critical forreinforcement learning, which is a cheap and principled w ay toautomatically optimise dialogue management. In this paper wepresent a no vel cluster -based technique for building user simula-tions which sho w varying , but complete and consistent beha viourwith respect to real users. W e use this technique to build usersimulations and we also introduce the S U P E R evaluation metricwhich allo ws us to evaluate user simulations with respect to thesedesiderata. W e sho w that the cluster -based user simulation tech-nique performs signiÞcantly better (at P < 0.01 ) than decisionsmade using either the one most likely action or a random base-line. The cluster -based user simulations reduce the average errorof these other models by 53% and 34% respecti vely .Index T erms : spok en dialogue, user simulation, evaluation met-rics, reinforcement learning, dialogue strate gies