Answers are grounded in this paper and cite their sources.
We evaluate six flagship Large Language Models (LLMs) (spanning prior- and current-generation systems across the ChatGPT, Claude, and Gemini families) on Pali-to-English translation for selected suttas from the Majjhima Nikaya. We introduce an automated pipeline that segments scripture into context-linked JSON, enforces format-faithful outputs via prompt engineering, and closes the loop with artificial intelligence (AI)-based quality assessment using the Generative Pretrained Transformer (GPT) Estimation Metric Based Assessment (GEMBA) in both no-reference (NR) and human-reference modes, alongside METEOR, TER, and BERTScore. Results show that modern LLMs can produce high-quality translations of ancient Pali: GPT-5 is the most consistent on NR-GEMBA (fewest sub-80 rated lines), while Claude 4 Sonnet aligns most closely with the human reference on traditional metrics; a compact head-to-head matrix further reveals a line-wise edge for Claude 3.5 Sonnet in pair-wise wins. Inter-evaluator agreement is moderate-to-high. Overall, the workflow can broaden access to Buddhist scripture while leaving final interpretive authority with human scholars, and it provides a scalable template for evaluating AI translations of ancient texts.
No takes yet. Share an insight, caveat, or question.
Phophichit et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: