What question did this study set out to answer?

The study aims to synthesize trends and developments in automated writing evaluation (AWE) research over the last decade.

March 6, 2026Educational Technology Research and Development1 citationsOpen Access

A meta-synthesis of automatic writing evaluation research: trends and developments over a decade

WTWen-Ta Tseng SCSufen ChenNational Taiwan University of Science and Technology TDThi Hue DongNational Taiwan University of Science and Technology

Key Points

The study aims to synthesize trends and developments in automated writing evaluation (AWE) research over the last decade.
Synthesis of eleven systematic reviews and meta-analyses on AWE research from 2015 to 2025.
Evaluation of selected reviews using the Many-Facet Rasch Model to assess study quality.
Utilization of narrative approach and text mining analysis for data synthesis.
Shift observed from rule-based AWE systems to AI-driven tools over three decades.
Support for AWE's effectiveness in surface-level writing aspects, such as grammar and spelling.
Identified limitations of AWE in enhancing high-order writing skills like argumentation.

Abstract

Abstract Automated writing evaluation (AWE) technologies have emerged as promising tools that streamline the feedback process and strengthen students’ writing skills. This meta-review synthesized eleven systematic reviews and meta-analyses on AWE research published from 2015 to 2025. Before the main analyses, all selected reviews were evaluated using Many-Facet Rasch Model (MFRM) to determine the study quality. Next, syntheses methods employed narrative approach and text mining analysis. The results suggested the shift from rule-based AWE system to AI-driven AWE tools over three decades. The synthesized findings from meta-analyses supported the effectiveness of AWE on surface-level writing (e.g., grammar, spelling) but highlighted its limitations in improving high-order level of writing (e.g., argumentation). Further, drawing on moderator analyses, educational levels and duration deserve attention in the implementation of AWE. Finally, persistent challenges, future research directions, and practical pedagogy were also identified and discussed. Overall, the present meta-synthesis study supports the potential value of AWE as an adjunct tool rather than a replacement for human feedback in writing instruction.

Ask AI

Helpful

Bookmark

View Full Paper