Model-induced hallucination, in which language models produce content that is factually inaccurate or contextually inconsistent, presents a substantial obstacle to dependable natural language comprehension. To ensure models can be trusted in applications such as question answering, summarization, and dialogue systems, it is important to test for hallucination across multiple benchmarks. Current approaches mostly use single-benchmark assessment or basic reference-based comparison, which don’t always capture small inconsistencies or errors that occur across different contexts in model outputs. These restrictions make it harder to find hallucinations and make it harder to create more reliable language models. To solve these problems, we provide a Cross-Benchmark Hallucination Detection approach called Reference-Based & Contextual Consistency Analysis (CB-HD). CB-HD measures model outputs against a variety of language understanding benchmarks using both reference-based comparisons and checks for consistency across contexts. The framework measures the degree of hallucinations by finding both factual errors and contradictions in the generated material. This provides a complete picture of the model’s reliability. The suggested strategy can be used for NLP tasks that require substantial information, such as automated question answering and summarization, to improve the model and reduce hallucinated content. Experimental results show that CB-HD can find hallucinations in many datasets, which makes the language model outputs more factually accurate and contextually coherent.
Mishra et al. (Thu,) studied this question.