Identification of Writing Strategies in Educational Assessments with an Unsupervised Learning Measurement Framework

This study proposes a framework that leverages natural language processing and unsupervised machine learning techniques to measure, identify, and classify examinees’ writing strategies. The framework integrates three categories of writing strategies (text complexity, evidence use, and argument structure) to identify the characteristics of examinees’ writing. Additionally, a measurement model is used to calibrate examinees’ writing proficiency. An empirical example is presented to demonstrate the performance of the framework. The data comprise 430 Grade 8 examinees’ responses to English Language Arts (ELA) assessments in the United States. Using K-means clustering, distinct patterns were identified in each category. The one-parameter logistic measurement model was applied to estimate examinees’ writing proficiency. Analyses revealed significant effects of text complexity and evidence use on writing proficiency, while argument structure was not significant. This study has implications for writing instruction and assessment design that highlight the point that effective writing is not simply a matter of isolated skill acquisition, but rather the coordinated implementation of complementary strategies, a finding that supports cognitive developmental theories of writing.