Introduction A considerable number of students enter secondary school with fundamental gaps in basic mathematical skills, particularly in their understanding of operations related to multiplication and division. The conceptual understanding of operations - operation sense - is the ability to relate situations (e.g., word problems) to mathematical-symbolic notations (e.g., calculations, equations) and vice versa and it is crucial for further learning in mathematics. In order to provide adaptive support for operation sense, the understanding or the lack thereof must be systematically assessed. Therefore, a sensitive assessment instrument specifically tailored to the operations of multiplication and division is needed, which can be used for a focused diagnosis and for evaluation of specific interventions in this domain. Thus, we developed an instrument for assessing multiplicative operation sense, which represents theoretically grounded levels of understanding operations in multiplicative situations. Methods The test was empirically validated in a pilot study ( N = 66). Item responses were analyzed using general linear mixed models to investigate differences in solution rates across levels and to estimate the proportion of variance in item difficulty explained by the theoretically derived levels. Based on the pilot results, three items were revised and additional shortcomings of the study design were addressed in a main study ( N = 464). Results In the pilot study, general linear mixed models showed that - as expected - the estimated solution rate decreases with increasing level and that 86% of the variance in item difficulty can be explained by the four theoretically derived levels of multiplicative operation sense. The main study showed 94% explained variance and significant mean differences between all levels. Discussion These findings support the validity of the instrument for assessing multiplicative operation sense and its usefulness for both research and practice.
Schultheis et al. (Wed,) studied this question.