Investigating Hallucination Tendencies of Large Language Models in Japanese and English

Abstract The increasing reliance on artificial intelligence for natural language processing has brought to light the issue of hallucinations in language models, where models generate content that appears plausible but is factually incorrect. Exploring the comparative hallucination tendencies in Japanese and English reveals significant differences, highlighting the importance of understanding language-specific challenges in model performance. A rigorous methodology was employed to quantify the frequency and severity of hallucinations, with comprehensive data collection from diverse sources in both languages. Quantitative analysis indicated a higher propensity for hallucinations in Japanese responses, attributed to the complex syntactical and contextual structures of the language. Qualitative examples provided concrete illustrations of the errors encountered, demonstrating the impact of linguistic and cultural factors. The findings emphasize the necessity for more linguistically diverse and contextually rich training datasets, along with advanced fact-checking mechanisms, to improve the reliability of language models. The study's implications extend to the development of tailored strategies for enhancing model accuracy across different languages, contributing to the broader goal of creating more robust and trustworthy artificial intelligence systems for global applications.