Assessing Dimensionality of a Set of Item Responses‐Comparison of Different Approaches

This study compares the performance of three methodologies for assessing unidi‐mensionality: DIMTEST, Holland and Rosenbaum's approach, and nonlinear factor analysis. Each method is examined and compared with other methods on simulated and real data sets. Seven data sets, all with 2,000 examinees, were generated: three unidimensional and four two‐dimensional data sets. Two levels of correlation between abilities were considered:ρ= 3 and ρ= . 7. Eight different real data sets were used: Four of them were expected to be unidimensional, and the other four were expected to be two‐dimensional. Findings suggest that all three methods correctly confirmed unidimensionality but differed in their ability to detect lack of unidimensionality. DIMTEST showed excellent power in detecting lack of unidimensionality; Holland and Rosenbaum's and nonlinear factor analysis approaches showed good power, provided the correlation between abilities was low.