Hello, It's GPT-2 -- How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems

Data scarcity is a long-standing and crucial challenge that hinders quick of task-oriented dialogue systems across multiple domains: -oriented dialogue models are expected to learn grammar, syntax, dialogue, decision making, and language generation from absurdly small amounts task-specific data. In this paper, we demonstrate that recent progress in modeling pre-training and transfer learning shows promise to overcome problem. We propose a task-oriented dialogue model that operates solely on input: it effectively bypasses explicit policy and language generation. Building on top of the TransferTransfo framework (Wolf et al. , 2019) generative model pre-training (Radford et al. , 2019), we validate the on complex multi-domain task-oriented dialogues from the MultiWOZ. Our automatic and human evaluations show that the proposed model is on with a strong task-specific neural baseline. In the long run, our approach promise to mitigate the data scarcity problem, and to support the of more engaging and more eloquent task-oriented conversational.