Identifiability conditions in cognitive diagnosis: Implications for Q‐matrix estimation algorithms

The Q‐matrix of a cognitively diagnostic assessment (CDA), documenting the item‐attribute associations, is a key component of any CDA. However, the true Q‐matrix underlying a CDA is never known and must be estimated—typically by content experts. However, due to fallible human judgment, misspecifications of the Q‐matrix may occur, resulting in the misclassification of examinees. In response to this challenge, algorithms have been developed to estimate the Q‐matrix from item responses. Some algorithms impose identifiability conditions while others do not. The debate about which is “right” is ongoing; especially, since these conditions are sufficient but not necessary, which means viable alternative Q‐matrix estimates may be ignored. In this study, the performance of Q‐matrix estimation algorithms that impose identifiability conditions on the Q‐matrix estimate was compared with that of estimation algorithms which do not impose such identifiability conditions. Large‐scale simulations examined the impact of factors like sample size, test length, attributes, or error levels. The estimated Q‐matrices were evaluated for meeting identifiability conditions and their accuracy in classifying examinees. The simulation results showed that for the various estimation algorithms studied here, imposing identifiability conditions on Q‐matrix estimation did not change outcomes with respect to identifiability or examinee classification.