Combining the fine-tuned GPT model and linguistic complexity indices in assessing second language (L2) Spanish writing

Automated Writing Evaluation (AWE) is increasingly leveraging Large Language Models (LLMs), but research on fine-tuned models for L2 Spanish and their relationship with linguistic complexity (LC) remains limited. This study evaluates three models for assessing a large corpus of L2 Spanish writing: (1) a fine-tuned Generative Pre-trained Transformer (GPT)-4.1 model, (2) a model using 87 LC indices, and (3) a hybrid model integrating both. Results demonstrate that the hybrid model achieved the highest reliability and accuracy (QWK = 0.93), significantly outperforming both the GPT model alone (QWK = 0.88) and the LC model (QWK = 0.83). The integration of LC indices particularly improved the accuracy for assessing Intermediate-level texts. These findings suggest a synergy where explicit linguistic features provide analytical precision that enhances the holistic judgments of the LLM. The study advocates for a hybrid approach to AWE that combines generative AI with linguistic analysis to create more accurate, valid, and interpretable assessment tools.