Improving Contextual Language Models for Response Retrieval in Multi-Turn Conversation

As an important branch of current dialogue systems, retrieval-based chatbots leverage information retrieval to select proper predefined responses. Various promising architectures have been designed for boosting response retrieval, however, few researches exploit the effectiveness of the pre-trained contextual language models. In this paper, we propose two approaches to adapt contextual language models in dialogue response selection task. In detail, the Speaker Segmentation approach is designed to discriminate different speakers to fully utilize speaker characteristics. Besides, we propose the Dialogue Augmentation approach, i.e., cutting off real conversations at different time points, to enlarge the training corpora. Compared with previous works which use utterance-level representations, our augmented contextual language models are able to obtain top-hole contextual dialogue representations for deeper semantic understanding. Evaluation on three large-scale datasets has demonstrated that our proposed approaches yield better performance than existing models.