Replication materials evaluate AI grading reliability in handwritten exams, suggesting high accuracy.
This repository contains the replication materials for a peer-reviewed study evaluating the reliability of Claude Code (Claude Opus 4.6, agentic deployment) for grading handwritten university exam papers. The study compared AI-assigned grades against blind human grades on twenty consented student papers (Pearson r = 0.97), tested grading consistency across two independent runs one week apart on a separate fifteen-paper sample (ICC(2,1) = 0.96), and coded the accuracy of AI-generated feedback (92.8% diagnostic accuracy, 97.8% confirmation accuracy). The exam was the mid-term assessment for CPIT285 Computer Graphics, held at King Abdulaziz University in April 2026, with sixty consenting students. Deposited materials include the rubric files used by the AI grader, the Claude Code configuration, anonymised grade reports and flag reports for all sessions, the analysis data files (post-reconciliation), R and Python analysis scripts, runtime environment logs, delivery scripts, and the final figures from the manuscript. Original student exam papers and identifiable data are not deposited, in accordance with the institutional ethics framework.
No takes yet. Share an insight, caveat, or question.
Rayed AlGhamdi (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: