RAR-Agent: A Rationale-Augmented Retrieval Framework for Legal Question Answering

Key Points

• The study aims to enhance the reliability and effectiveness of legal question answering frameworks by addressing issues of knowledge obsolescence and hallucination in language models.
• Developed the RAR-Agent framework utilizing Chain-of-Thought and Rationale-based query formulation techniques.
• Implemented Reciprocal Rank Fusion and Reranker-based filtering mechanisms to address vocabulary discrepancies.
• Created the KL-BQA benchmark to precisely measure the factual accuracy of the agent.
• Evaluated performance on the KL-BQA and KL-RQA benchmarks.
• RAR-Agent demonstrated improved performance in retrieving relevant legal information.
• Significantly mitigated hallucination issues compared to previous models.
• Achieved superior accuracy levels on both KL-BQA and KL-RQA benchmarks.