Evaluating the validity and reliability of AI-generated rubrics for EAP assessments: A case study in a transnational university

This case study examines the effectiveness of AI-generated rubrics for English for Academic Purposes (EAP) assessments at a transnational university, focusing on analytic rubrics produced by ChatGPT. Rubrics are essential for structured grading, enhancing clarity and consistency ( Andrade & Du, 2005 ; Campbell, 2005 ). However, creating detailed descriptors can be time-consuming. The study evaluates AI-generated rubrics in promoting grading consistency among instructors during standardization meetings, where they scored sample papers. The analysis focused on the validity and reliability of the rubrics, guided by Higher Education Quality Council of Ontario (HEQCO) standards (McKeown & Biss, 2018). Findings provide insights and recommendations for integrating these rubrics into EAP assessments. This article was published open access under a CC BY-NC-ND licence: https://creativecommons.org/licenses/by-nc-nd/4.0/ .