Evaluating Chinese Language Comprehension of Generative AI Models Based on Korean Prompts : Focusing on the 2026 College Scholastic Ability Test Chinese Questions

This study evaluates the Chinese language capabilities of 13 generative AI models (7 English-based, 6 Chinese-based) using the 2026 Korean College Scholastic Ability Test. Employing zero-shot Korean prompts, we analyzed 30 items across five dimensions: knowledge, reasoning, language comprehension, multimodal understanding, and generalization. Beyond accuracy rates, we examined response quality through Grice’s Maxim of Quantity and efficiency indices. Results show Deepseek achieved 100% accuracy, while Thinking-mode models demonstrated superior reasoning with concise explanations. Chinese-native models exhibited translation patterns, whereas English-based models provided more natural Korean responses. Findings indicate that effective educational application requires strategic model selection based on pedagogical objectives. Enhancing teachers’ AI literacy in model selection and output verification is essential for successful integration.