Answers are grounded in this paper and cite their sources.
Open-ended historical questions can elicit different responses from a large language model when only the input language changes. We tested this with 200 questions about the Great Leap Forward and related modern Chinese history, written in English, Chinese, and Japanese. ChatGPT and Gemini generated 1,200 answers. We compared each answer with a reference in the same language using Emotion Risk (ER) and Embedding Distance (ED), combined these measures in a Final Score, and measured score variation across languages. Both models had their highest mean Final Scores for Japanese questions: 39.04 for ChatGPT and 39.68 for Gemini. We then split the questions by ID and fine-tuned Qwen2.5-1.5B-Instruct with supervised fine-tuning (SFT) and Direct Preference Optimization (DPO). On 90 held-out language-level samples, mean Final Score fell from 34.55 in the original model to 34.36 after SFT and 31.88 after DPO. SFT gave the lowest mean Cross-Language Disparity (5.12, compared with 6.90 originally and 5.91 after DPO). These results show measurable language-conditioned differences in this historical question set and a reduction in the proposed response-risk score after training. The automatic metrics do not establish factual accuracy or historical bias on their own.
No takes yet. Share an insight, caveat, or question.
Katelyn Chen (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: