Developing Meaning-Based Complexity Indices Based on Cognitive-Functional Connectives and Validating Their Predictive Utility in Automated Writing Evaluation

Abstract This study develops and validates three meaning-based complexity indices—Academic Diversity (ADiv), Academic Weight (AWeight), and Academic Rate (ARate)—to quantify academic vocabulary usage in native Korean undergraduate students’ argumentative writing and assess their predictive value in writing evaluation. Drawing on a large-scale national writing assessment corpus comprising 2,429 essays from two academic prompts, the study investigates how these indices reflect cognitive depth and discourse competence in L1 academic writing.Regression analyses reveal that AWeight, which captures the semantic sophistication of academic terms based on graded difficulty, exhibits a consistently positive relationship with writing scores. Contrary to prior assumptions of an inverted-U shape, no detrimental effects of “excessive complexity” were observed within the actual learner range. AWeight also emerged as the most effective diagnostic indicator for distinguishing high- and low-scoring writers, with a potential benchmark range identified between 20–25 points.While integrated models incorporating all three indices yielded only marginal improvements in predictive power (R² = 0.0574), clustering analyses revealed distinct lexical strategy types, highlighting variation in how native speakers deploy academic vocabulary. Notably, a subgroup exhibiting “inappropriate complexity”—high lexical sophistication but low scores—suggests that lexical complexity alone is insufficient without discourse-level alignment.These findings underscore the gatekeeping function of academic vocabulary complexity in automated writing evaluation (AWE) systems and call for multi-dimensional assessment frameworks that integrate semantic depth, discourse appropriateness, and strategic deployment. The study contributes theoretically by reframing lexical complexity as a context-sensitive, functional construct, and practically by proposing pedagogically interpretable indices for AWE and learner diagnostics in native academic writing contexts.