Knowledge of Solution Strategies and IRT Modeling of Items for Transitive Reasoning

Componential item response theory (CIRT) is presented as a model-oriented approach to studying processes and strategies underlying the incorrect/correct responses to cognitive test tasks. CIRT is contrasted with a data-oriented approach in which verbal explanations for incorrect/correct responses are collected during the test phase and incorporated in the scoring. Alternatively, the psychologically meaningful data are modeled by unidimensional item response theory models. Verbal explanations for each examinee and task were collected from transitive reasoning tasks in addition to the incorrect/correct responses. Two datasets were compiled, one reflecting the common incorrect/correct scoring and one showing whether a deductive strategy had been used to produce a correct response. The Mokken model of monotone homogeneity, the partial-credit model, and the generalized one-parameter logistic model were used to analyze both polytomous datasets. Results showed that combining knowledge of solution strategies with IRT modeling produced a useful unidimensional scale for transitive reasoning.