ABSTRACT Background The use of recent artificial intelligence (AI) chatbots, such as chat generative pretrained transformer (ChatGPT), in second language (L2) writing may face criticism for potentially promoting plagiarism and raising ethical concerns. However, such tools can effectively provide guided suggestions for improving outlines, content, and organisation. Additionally, EditGPT, an extension that tracks changes made by ChatGPT, provides immediate, direct corrective feedback, thereby enhancing L2 learners' control over their writing process. Objectives The aim of this study was to compare the quality of L2 learners' writing (LW), ChatGPT‐proofread writing (PW), and learners' ChatGPT‐supported revisions (RW) by analysing human raters' evaluations, assessing linguistic complexity, and identifying error types. Methods A total of 40 university students majoring in English education, all of whom were English as a foreign language (EFL) learners and pre‐service teachers, participated in this study. Based on significant differences in their TOEIC scores, they were divided into two groups. The secondary‐school pre‐service teachers were classified as skilled learners, while the elementary‐school pre‐service teachers were identified as less skilled learners. Participants completed a guided writing task supported by ChatGPT for idea generation and essay revision. They interacted directly with ChatGPT using standard prompts for brainstorming and outlining and received automated feedback on editing and proofreading through the EditGPT extension. Results When LW and RW were compared, human raters found significant improvements in language use but not in content and organisation . The revisions demonstrated greater linguistic complexity, including increased use of academic words, improved lexical sophistication, and more varied sentence structures. Regarding language errors, 86.17% of the errors in LW were successfully rectified. However, a closer breakdown revealed that certain error types—particularly word choice (38.9% remaining) and sentence structure (35.4% remaining)—remained relatively unresolved. Addressing these lingering errors in both PW and RW often required judgements of contextual appropriateness, especially when attempting to preserve the learners' intended meanings and nuanced expressions. As a result, the overall error resolution rate dropped to 77.02% in RW. Conclusions The structured AI‐mediated revision environment was associated with short‐term improvements in the linguistic quality of L2 writing within a controlled session. These effects reflect product‐level changes and cannot be attributed to individual writing process components. While AI‐supported revision reduced many surface‐level errors, limitations remained for context‐sensitive language use. Further longitudinal research is needed to determine whether these changes support sustained writing development.
Chon et al. (Sun,) studied this question.