Confirmatory Methods Comparison for Measuring Latent Multidimensional Structures in Assessments

Confirming assumed latent multidimensional structures is a crucial step in developing assessment tools. This study compares three confirmatory methods (including CFA, CMIRT, and GDINA) and corresponding model-fit indices regarding confirming true structures over misspecified structures, based on 162,000 simulated binary data under between-item, within-item, bifactor, and two-tier structures. The results show that for model-fit indices, in CFA, AIC and BIC outperform others for two-tier and the other structures, respectively; in CMIRT, BIC and AIC perform better than others for between-item and the other structures, respectively; in GDINA, SRMSR and AIC outperform others for two-tier and the other structures, respectively. Regarding methods comparison, for between-item and within-item structures, CFA and CMIRT can be trusted more than GDINA; for two-tier structures, GDINA and CMIRT perform better than CFA; for bifactor structures, GDINA outperforms others. Given that none of them is optimal, we recommend using multiple methods as a kind of sensitivity analysis.