Exploring AI-assistance in L2 Chinese writing with standardized assessment tasks

While many studies have addressed the benefits of technology-assisted L2 writing, limited research has delved into how generative artificial intelligence (GAI) supports students in completing their writing tasks in Mandarin Chinese. In this study, 26 university-level Mandarin Chinese foreign language students completed two writing tasks on two separate days. The writing prompts were from the AP Chinese Language and Culture Exam free response questions. Students were permitted to use ChatGPT for assistance with only one of the given prompts each day. After data analysis, writing scores were compared with and without the use of ChatGPT to discern which aspects of ChatGPT students relied on the most. To maintain the validity and reliability of the scores, students' writing samples were independently evaluated by two professional raters, yielding a high degree of inter-rater reliability. The results indicate that utilizing ChatGPT for assistance consistently resulted in higher scores. Additionally, students primarily sought assistance from ChatGPT for vocabulary-related and translation-related questions. Quantitative and qualitative data were used to analyze students’ perceptions of using GAI to support the students in completing the writing tasks. The current study seeks to bridge existing knowledge gaps and offer empirical data on how GAI tools, like ChatGPT, can develop students’ L2 writing within the Mandarin Chinese language learning context.