On Deep Learning Approaches to Automated Assessment: Strategies for Short Answer Grading

The recent increase in the number of courses that are delivered in a blended fashion, before the effect of the pandemic has even been considered, has led to a concurrent interest in the question of how appropriate or useful automated assessment can be in such a setting. In this paper, we consider the case of automated short answer grading (ASAG), i.e., the evaluation of student answers that are strictly limited in terms of length using machine learning and in particular deep learning methods. Although ASAG has been studied for over 50 years, it is still one of the most active areas of NLP research as it represents a starting point for the possible consideration of more open ended or conversational answering. The availability of good training data, including inter alia, labelled and domain-specific information is a key challenge for ASAG. This paper reviews deep learning approaches to this question. In particular, deep learning models, dataset curation, and evaluation metrics for ASAG are considered in some detail. Finally, this study considers the development of guidelines for educators to improve the applicability of ASAG research.