Algorithmic Item Generation of Spatial Ability Items: A Cognitive Design System Approach

Item generation based on the item model approach can efficiently produce a large number of items at a low cost, but fails to guarantee and evaluate the construct validity of the generated items. To achieve more scientific item generation and validity testing for spatial ability, this study developed an item generation algorithm for spatial ability based on the cognitive design system (CDS) approach and evaluated the construct validity of the generated items. A total of 624 mental rotation items were efficiently generated via the algorithm and administered to 3846 primary and secondary school students. The results indicate that the four construct-relevant design variables formulating the algorithm all significantly affected item difficulty, accounting for 72.6% of the variance in item difficulty. Furthermore, the nomothetic span analysis revealed that scores of the developed mental rotation test significantly correlated with scores from tests measuring visuospatial working memory, paper folding, and mathematics. In contrast, statistically significant correlations were not detected between the mental rotation test and the tests measuring cognitive processing speed and Chinese language. These findings provide initial evidence supporting both the rationality of utilizing the CDS approach to generate mental rotation items and the construct representation of the generated items, while offering tentative empirical observations regarding their nomothetic span.