Key points are not available for this paper at this time.
ESL/EFL students need to enhance writing skills to pass various language tests. To help teachers reduce their teaching load and to give students faster feedback, several commercial AWE (automated writing evaluation) tools are available. Although these innovative tools have become popular, few empirical studies have examined the quality of feedback generated by these systems. There is a pressing need to better understand the strengths and weaknesses of these widely used AWE systems. In this paper, we examined the grammar feedback provided by two well-known systems, Vantage My Access and ETS Criterion. The purpose of this paper is to uncover the strengths and weaknesses of the feedback provided by these two systems. Automated feedback messages on 269 student essays were collected and analyzed. The results showed that these two systems can provide about 30 different types of feedback messages. However, My Access provided many false alarm messages. Criterion performed much better than My Access, but it still failed to deal with several common errors related to word order, modals, tenses, collocations, conjuncts, word choice, and pronouns. It is clear that the two AWE systems do not provide satisfactory help to ESL students. Teachers and students who use these new tools should be aware of these problems and limitations.
Chen et al. (2009) studied this question.