Improving Response Quality in Conversational AI: The Role of Smart Answer Encoding in Retrieval Augmented Generation

In tasks like generative question answering, where models retrieve information from external knowledge sources, the quality of generation depends heavily on the relevance of retrieved passages. Current models focus excessively on label-relevant passages rather than question-relevant ones. This study addresses this gap by integrating Dense Knowledge Similarity (DKS) and Retriever as Answer Classifier (RAC) for smarter answer encoding. Our approach outperforms existing methods in open domain question answering (MSMARCO) and conversation (Wizard of Wikipedia) datasets. On the MSMARCO development set, our best model shows a 12.1% relative improvement in Recall@1 and a 4.5% relative improvement in BLEU-4 over the baseline model. On the KILT-WoW leaderboard, our best model exhibits an 8.9% relative improvement in R-Precision and a 13.3% relative improvement in KILT-RL compared to the baseline model.