How Well Do Generative LLMs Translate Chinese Political Texts into English? A Multi-Metric Evaluation with Human Judgment

Taking the 2026 New Year Address by China’s president as its research corpus, this study selects ten representative sentences to evaluate English translations produced by four generative large‑language models: Gemini, Doubao, DeepSeek and ChatGPT. It adopts a mixed-method evaluation framework that integrates three widely‑used automatic evaluation metrics, namely METEOR, BERTScore and COMET, together with expert human scoring. On this basis, the paper makes a comparative analysis of the consistency between scores yielded by different automatic evaluation metrics and human expert judgments. The experimental results indicate that these automated metrics differ substantially in their respective evaluation priorities, and neural‑network‑based evaluation metrics achieve relatively higher consistency with human ratings. Within the scope of the present corpus, the comprehensive translation performance of the four large‑language models is ranked in descending order as follows: ChatGPT, Doubao, DeepSeek, and Gemini.