A Family of Sequential Item Response Models for Multiple-Choice, Multiple-Attempt Test Items

We consider a test which allows students to attempt a multiple-choice question multiple times (i.e., multiple attempts). The most extreme form of multiple attempts is the answer-until-correct (AUC) procedure, which allows subjects to respond to a multiple-choice item until the correct answer option is selected. Previous research has demonstrated that multiple-attempt procedures such as AUC could potentially enhance learning, increase reliability, and discourage guessing. However, for multiple-choice items, guessing is still non-ignorable. Traditional models of sequential item response theory (SIRT) could model multiple-attempt procedures but fail to take guessing into account. The purpose of this study is to propose SIRT models for multiple-choice, multiple-attempt items (SIRT-MM). First, we defined a family of SIRT-MM models by imposing or lifting constraints on certain parameters to account for the idiosyncrasies of items, answer options, and examinee behavior, and explained how these models could improve person parameter estimates by taking into account partial (mis)information of examinees. Second, we conducted model comparisons between the SIRT-MM models, the graded response model, and the nominal response model. We showed that when data are generated from SIRT-MM models, AIC and BIC indices properly select SIRT-MM models as the best fitting model. Third, we discussed how the item and person parameters can be estimated, and evaluated item and person parameter recovery of SIRT-MM models under different conditions through a simulation study.