Machine learning-based item difficulty prediction model using item text features

This study investigates the difficulty of English reading passages in the College Scholastic Ability Test (CSAT) in South Korea, analyzing corpus linguistic features from 2015 to 2023 using Coh-Metrix software. A notable innovation of this research is the application of the Random Forest algorithm, which overcomes the limitations of traditional regression analysis, such as multicollinearity and overfitting. Focusing on textual complexity, the analysis utilizes 108 explanatory variables to predict the difficulty of CSAT English items. Rigorous methods, including grid search and cross-validation, were employed to ensure the reliability of the results. The Random Forest regression model demonstrated high predictive accuracy. Crucial variables influencing the difficulty of items were identified, providing valuable insights for educators and policymakers to improve CSAT preparation strategies. This research marks a significant advancement in language assessment by combining corpus analysis with machine learning algorithms. It not only enhances the prediction of test difficulty but also supports students and teachers with effective, tailored preparation strategies to address the varied challenges encountered in English public education.