What question did this study set out to answer?

This study aims to develop a method for detecting hallucinated facts in generative large language models without relying on external databases.

January 1, 2023Open Access

SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models

View Full Paper

Ask AI

Bookmark

Authors

Randomized trial investigates hallucination detection in generative models, suggesting a new approach for fact-checking.

This study aims to develop a method for detecting hallucinated facts in generative large language models without relying on external databases.
Proposed SelfCheckGPT, a sampling-based approach for fact-checking black-box models.
Generated responses from GPT-3 using the WikiBio dataset and manually annotated their factuality.
Compared SelfCheckGPT to baseline models using various performance metrics, including AUC-PR scores.
SelfCheckGPT effectively detects non-factual and factual sentences within responses.
Achieved significantly higher AUC-PR scores in sentence-level hallucination detection compared to baseline methods.
Showed improved correlation scores in assessing the factuality of generated passages.

Manakul et al. (2023) studied this question.

View Full Paper

Ask AI

Bookmark