Answers are grounded in this paper and cite their sources.
Background Giving meaningful feedback on assignments in large introductory-level mathematics courses is a challenge. The emergence of “out of the box” Large Language Models (e.g., ChatGPT, Gemini) has created a potential solution to this challenge, where the AI tool can be trained to give constructive feedback on student work. However, it is unclear whether these tools are sufficiently accurate, efficient, and trustworthy. The current study investigates the use of a bespoke AI tool to assist in giving feedback to students in two introductory mathematics courses that enroll over 400 students each and are characterized by high attrition. Results We evaluated the similarities in grades assigned by the AI tool and human graders, as well as student and grader perceptions of the feedback given by both the AI tool and human graders. We found that the tool was fairly accurate, but that perceptions of the utility of its feedback varied by condition. When evaluated as a stand-alone grading agent, the AI tool was significantly undervalued relative to human graders—both student peers and the professor. Conclusion Three themes that characterize our findings: student trust is key, the professor is irreplaceable, and graders and AI tools must be carefully trained to ensure accuracy, transparency, and trust. We conclude with a call to exercise caution in using these tools in grading assistance, particularly with respect to maintaining student trust and transparency in grading.
No takes yet. Share an insight, caveat, or question.
Saltskår et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: