An Exploratory Study on Factors Influencing Rasch Item Difficulty and Item Proportion Correct Using LLTM and LLTM-R models: Focusing on 3R’s Math Test for 3rd Grade

This study aimed to predict the Rasch item difficulty and the proportion of correct responses for the 3R's math tests in the Diagnostic Tests of Basic Skills using the Linear Logistic Test Model (LLTM) and the Linear Logistic Test Model with error (LLTM-R). Q-matrices with 12 item attribute variables were derived, and the national samples were collected in 2018, 2019, 2022, and 2023. Findings reveal that, firstly, items requiring problem-solving with pictures and carrying/borrowing were challenging, while attributes like solving by reading numbers, single-digit, and constructed-response were prevalent. Secondly, correlations between Rasch difficulties and LLTM estimates varied between 0.67-0.79, but approached 1.0 with LLTM-R. Thirdly, after predicting item responses with LLTM-R, correlations between the observed proportion of correct responses and the predicted proportions of correct responses were approximately 0.74 to 0.76. The percentages of items with less than a 5% difference between observed proportion of correct response and predicted proportion of correct responses over four years ranged from 68% to 92%, and there were no items with the difference of more than 10%.