An AI-Powered Sentence-BERT Framework for Automated Assessment of Open-Ended Responses: A Multi-Model Evaluation and Response-Length Bias Analysis

The automated assessment of open-ended responses remains challenging due to the complexity and subjective nature of human grading. A further methodological challenge is determining which pretrained Sentence-BERT model and scoring configuration can provide reliable agreement with human assessment while maintaining computational efficiency. This study presents an AI-powered framework based on Sentence-BERT models for automated grading of short-answer responses, focusing on model performance, reliability, response-length sensitivity, and hybrid scoring. A custom dataset of student responses to open-ended questions, annotated by three independent human raters, was developed and pre-processed. Five Sentence-BERT models, all-MiniLM-L12-v2, all-mpnet-base-v2, stsb-roberta-large, paraphrase-MiniLM-L12-v2, and distilbert-base-nli-stsb-mean-tokens, were evaluated using semantic similarity scoring and multiple metrics, including Pearson and Spearman correlations, Cohen’s Kappa, MAE, RMSE, and ICC. Among the evaluated models, all-MiniLM-L12-v2 provided the most favorable balance of grading performance, reliability (ICC = 0.527), and computational efficiency. Human ratings showed stronger dependence on response length (Pearson r = 0.663) than the evaluated AI models (r = 0.338–0.474), indicating reduced sensitivity to verbosity. Weight-sensitivity analysis showed that the 25% semantic–75% rubric configuration had the lowest continuous-score MAE (1.8499) and RMSE (2.3980) among the tested configurations, supporting the complementary contribution of holistic semantic similarity and rubric-based concept coverage. The selected model and hybrid scoring mechanism were implemented in a web-based prototype providing automated scores and interpretable feedback while retaining instructor review.