Modeling Individual Differences in Numerical Reasoning Speed as a Random Effect of Response Time Limits

If numerical reasoning items are administered under time limits, will two dimensions be required to account for the responses, a numerical ability dimension and a speed dimension? A total of 182 college students answered 74 numerical reasoning items. Every item was taken with and without time limits by half the students. Three psychometric models were fit to the data—one including no time-limit effect, one including a fixed time-limit effect, and one including a random effect of time limits. The latter model best fit the data, suggesting that a speed dimension, the random effect of time limits, is needed to account for time-limited responses. The estimated reliability of the Speed scores was .39. Despite this low reliability, Speed scores were correlated with American College Testing (ACT) math scores and response times. Speed scores added significantly to the ACT math score variance accounted for by the numerical reasoning dimension in the model. A within-person log-odds ratio interpretation of the Speed score is proposed. Possible methods of improving Speed score reliability and methods for studying the speed dimension are discussed.