Answers are grounded in this paper and cite their sources.
The use of artificial intelligence (AI) to support assessment and feedback in higher education has attracted growing interest, but concerns remain about the consistency and appropriateness of AI-generated grades in summative contexts. This study examined whether a combined calibration strategy involving a more explicit rubric and marking prompt could improve agreement between AI-assisted grading and educator judgement in a laboratory-based STEM assessment. We compared educator-assigned marks for 40 graduate Electrochemical Energy Storage Systems laboratory reports with marks generated by GPT-5 under pre- and post-calibration configurations, and examined agreement using complementary statistical and visual analyses. Following calibration, agreement with educator marks improved across measures of score correspondence and absolute agreement: the intraclass correlation coefficient increased from 0.818 to 0.928, the mean AI-educator difference decreased from +1.625 to +1.041 marks, and no post-calibration report differed from the educator mark by more than 10 marks. These findings suggest that the way assessment criteria, evidence requirements, and marking rules are specified can materially influence AI-human grading agreement. However, the revised rubric and prompt were introduced together, calibration drew on discrepancies observed in the same 40 reports used for post-calibration evaluation, and one educator's marks served as the reference. The results therefore support improved in-sample agreement under a human-supervised calibration strategy, rather than establishing grading validity or interchangeability for autonomous summative decisions.
No takes yet. Share an insight, caveat, or question.
Deepshikha et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: