Background Qualitative revision of the Cognitive Model (CM) is prone to subjective bias, while classical quantitative methods can only indicate model-data misfit without offering specific revision guidance. Therefore, inadequate CM fit remains a common challenge in Cognitive Diagnosis (CD) research. Purpose This study aims to determine the CM while avoiding both the loss of examination of disciplinary connotation due to pure data-driven approaches and the potential biases that may arise from pure qualitative research. Method Using Structural Equation Modeling (SEM) as platform, this study analyzed the valid response data from 752 students by leveraging information such as r and Modification Index (MI). Additionally, it incorporated the judgments of experts regarding the disciplinary connotations. Through these efforts, the preliminary CM for the content of “Composition and Structure of Substances” in middle school chemistry was revised. Result The revised CM achieved improved fit and aligned with disciplinary connotations. Conclusion This study employs a mixed-method SEM approach. It avoids two pitfalls. Pure data-driven methods tend to deviate from disciplinary connotations, and pure qualitative analyses are prone to subjective judgment. The resulting benefits are better data-model fit, ensured disciplinary connotations of the CM, and operational efficiency. Meanwhile, it also has drawbacks related to measurement error and sample generalizability.
Lin et al. (Mon,) studied this question.