Cross-model validation of large language models in English-to-Chinese consecutive interpreting assessment: a multi-dimensional comparison with trained human ratings

LLMs are increasingly deployed to score essays, interviews, translations, and professional certifications. Demonstrating the reliability and validity of LLM scoring is a prerequisite for its adoption in educational, professional, and credentialing settings. The present study empirically examined the rating fidelity of five Chinese LLMs—Qwen, DeepSeek, Doubao, Ernie, and Spark—relative to human judgment. The dataset comprised 312 performances produced by 39 undergraduate learners. Three human raters assessed the original audio recordings, whereas the LLMs evaluated manually verified Chinese transcripts together with the corresponding English source texts. Both groups used the same three-dimensional analytic rating scale comprising accuracy, fluency of delivery, and target language quality. The study investigated the alignment between human and LLM-generated scores for English–Chinese consecutive interpreting, employing rank-based correlation (Spearman’s ρ), absolute error metrics (MAE and RMSE), agreement analysis, and post-hoc calibration. Results revealed moderate but heterogeneous concordance across models and assessment dimensions (ρ = 0.369–0.733). Ernie achieved the highest rank-order consistency with human ratings (Spearman’s ρ), while Qwen and DeepSeek exhibited lower absolute error (MAE/RMSE) on total scores, indicating stronger convergence in score magnitude. However, none of the LLMs achieved sufficient absolute agreement to support the direct use of raw AI-generated scores in practice. Post-hoc calibration generally reduced systematic bias and absolute error, yet improvements in agreement varied across models and dimensions. These findings support the cautious use of transcript-based LLM scoring in low-stakes contexts, but do not justify direct substitution for trained human ratings in high-stakes assessment.