Examining the Impact of Feedback from Automated Writing Evaluation Tools on Learners’ Grammar Accuracy

Improving the writing abilities of language learners remains a significant challenge. Although research indicates that automated writing evaluation (AWE) tools can improve learners’ grammar accuracy and that learners find AWE feedback helpful, the specific impact of different types of feedback offered by these tools on this improvement remains unclear. This study investigated the effects of two AWE tools on learners’ grammar accuracy: Grammarly, which offers both metacognitive feedback and direct correction, and Ginger, which offers direct correction. Learners’ perceptions of using these tools were also explored. Three groups were recruited and randomly assigned as a control group and two experimental groups. All groups received teacher feedback on the writing content, while the experimental groups additionally received feedback from the two AWE tools, respectively. Four writing tasks were conducted, with the first serving as a pre-test and the third and fourth serving as a post-test and a delayed post-test, respectively. Following completion of all writing tasks, the experimental groups filled out a user survey to gather their thoughts on AWE. The analysis of learners’ written texts has shown that the AWE improved learners’ grammar accuracy in writing as they allowed learners to identify grammar-related errors and possibly learn from their mistakes. Additionally, both types of corrective feedback were effective in reducing grammar-related errors, particularly those related to rule-governed errors. Many learners suggested that the feedback provided by AWE was beneficial for their writing tasks, so they were willing to use them to revise their texts. Pedagogical implications were discussed.