Answers are grounded in this paper and cite their sources.
Validation of a formative assessment system improves feedback and grading accuracy in higher education, indicating effective AI integration.
This study presents the design and validation of a formative assessment system that automatically grades and provides feedback on open-ended responses in higher education using large language models (LLMs). Implemented locally through the Ollama platform with a customized course corpus, it generates an “ideal response” based on instructor criteria and evaluates student answers accordingly. Semantic filters and sequential tolerance thresholds ensure alignment between AI and human grading. A dataset of 450 exams (2022–2025) from three technical courses was used, with 360 for model optimization and 90 for validation. Performance metrics included mean difference, Pearson correlation, intraclass correlation coefficient (ICC), and Bland–Altman analysis. Results showed strong agreement (mean absolute difference = 4.5/100, r > 0.95, ICC ≈ 0.95) and negligible bias (±8/100). The system generated detailed, pedagogically coherent feedback and effectively filtered irrelevant or ambiguous responses. Overall, it provided accurate and consistent assessment, greatly reducing grading time and enhancing formative learning while preserving instructor oversight.
No takes yet. Share an insight, caveat, or question.
Torrecilla et al. (2025) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: