Fine-tuning-based automated scoring of mathematical constructed-response items: Focusing on the effects of training data size and task characteristics

This study aims to explore the feasibility of automated scoring of mathematical constructed-response items using the fine-tuning of a large language model (LLM).To this end, the performance of fine-tuning was analyzed by varying the amount of training data, and implications were drawn accordingly.First, constructed-response tasks in three distinct areas of mathematics were developed, student responses were collected, and expert scoring was conducted.The results of automated scoring performed by a publicly available base LLM-provided with items and scoring rubrics-were then compared with those obtained after fine-tuning.The results indicated that scoring performance improved as the amount of training data increased, with a substantial improvement observed when at least 30 training samples per score category were secured.Furthermore, the degree of performance improvement varied depending on the characteristics of the task.Based on these findings, implications for the educational application of fine-tuning in mathematics education were discussed.