How is the Korean language ability of generative artificial intelligence ChatGPT?: Analysis of error types in the results of solving items in the Korean language section of the 2024 CSAT

Using OpenAI’s ChatGPT to determine the Korean language ability of gener ative artificial intelligence, this study analyzes the results of solving items in the Korean language area of the 2024 CSAT as well as the types of errors found in the process. Therefore, the researchers designed prompts suitable for solving items and collected the basis for the noon judgment for each option based on the correct answer to the question and the content of the text from ChatGPT. Prior to the full-scale analysis, ChatGPT’s tendency toward correct answers for each items was examined, and the percentage of correct answers was confirmed to be higher in the reading area than in literature and in items requiring vocabulary, concepts, and factual understanding rather than inferential and critical understand ing. Additionally, ChatGPT’s error types for item solving were largely categorized into two types: one in which the basis for the noon judgment of the item was not found within the text, and the other in which the meaning of the text was incorrect ly constructed. Furthermore, in the latter case, ChatGPT was divided into cases in which only the accumulative meaning intervened in the process of constructing the meaning of the text to create the wrong meaning, or the wrong meaning was constructed by not connecting or incorrectly connecting the information within the text. The results of these studies are significant as they empirically analyze and exam ine the Korean language ability of generative artificial intelligence and suggest the role of readers and writers in this era of generative artificial intelligence. Based on this study, conducting a comprehensive comparative review of the Korean language capabilities of generative artificial intelligence in the future is necessary. Moreover, as the time difference between the Korean language phenomenon and Korean lan guage education has intensified owing to the recent development of technology, discourse related to generative artificial intelligence will need to be triggered and discussed in the future to support readers and writers in the ever-changing literacy environment.