Exploring Bert-Based Encoders for Sequence Classification and Multi-Task Learning in Dialogue Acts and Joint Intent-Slot Filling

Understanding user utterances accurately is a cornerstone of effective conversational AI systems.This paper explores the fine-tuning of the BERT language model to enhance its performance in two pivotal natural language understanding tasks: dialogue act sequence classification and joint intent-slot filling through a multi-task learning approach.We first develop a sequence classification model for dialogue acts, leveraging BERT's deep contextualized embeddings to improve the accuracy of identifying the communicative functions of user utterances.Additionally, we design a joint intent-slot filling model, where BERT is fine-tuned in a multi-task learning framework to simultaneously predict user intents and extract slot information.This approach ensures that the model benefits from shared representations, enhancing its capability to perform both tasks efficiently and effectively.Experimental evaluations on MmTravel (Myanmar Travel) corpus, which is travel domain human-human conversations dataset (consists of 80,000 utterances), demonstrate that our fine-tuned BERT models achieve state-of-the-art results in both dialogue act classification and joint intent-slot filling.The multi-task learning strategy not only improves performance metrics but also reduces the computational overhead compared to training separate models for each task.Our findings underscore the efficacy of fine-tuning BERT for complex dialogue understanding tasks, providing a robust solution for developing more accurate and responsive conversational AI systems.This study contributes to the importance of advanced sequence classification and multi-task learning techniques in enhancing the capabilities of conversational AI systems, paving the way for more accurate and context-aware dialogue understanding.