Dimensionality, bias, independence and measurement scale problems in latent trait test score models

It is argued that latent trait models used in the analysis of test scores, in particular the Rasch model, suffer from serious defects. They assume implicitly a measurement scale model for which little substantive justification seems to be available and they make the assumption of ‘local independence’ which enjoys little empirical support. Perhaps their most serious drawback, which is illustrated with the fixed effects' Rasch model, is that the individual ability estimates obtained are always biased. This bias is not constant but depends upon an individual's true ability, the number of items in the test and their difficulty values. Moreover, there is no way to obtain an unbiased estimate in the typical case when only one administration of a test is given to individuals. This result suggests that the use of such models needs to be critically examined, especially in connection with so‐called ‘item banks’. The paper also discusses the dimensionality of the latent space, and shows how multidimensional models may be specified.