Argumentative element effectiveness and writing quality across task types: A machine learning analysis

Previous research on argumentative writing has focused largely on whether argumentative elements are present, with less attention to their effectiveness. This study examines how element presence and effectiveness relate to holistic writing quality in independent and source-based writing. Using the PERSUADE 2.0 corpus, comprising independent essays ( n = 13,121) and source-based essays ( n = 12,875), we analyzed seven argumentative elements: lead, position, claim, evidence, counterclaim, rebuttal, and concluding summary. Random forest regression models compared predictions based on essay length, element presence, and effectiveness ratings, while Spearman correlations examined relationships among the effectiveness ratings of key elements. Across both tasks, incorporating effectiveness ratings improved prediction beyond models based on essay length and element presence, although essay length remained a strong predictor. Concluding summary ranked highest among argumentative features in both tasks; rebuttal and lead were also prominent within independent writing, and claim-related features within source-based writing. Effectiveness ratings were positively associated among key elements in both contexts, particularly among claims and evidence-related features, suggesting that effective realization tends to extend across multiple components of an argument. These findings support examining argumentative writing through both the presence and effectiveness of its elements, while recognizing task-associated patterns in predictive importance and relationships among effectiveness ratings. For writing instruction and assessment, this study highlights the value of attending to how effectively argumentative elements fulfill their rhetorical functions and how their effectiveness is interrelated.