A Peculiarity in Educational Measurement Practices

This essay discusses a peculiarity in institutionalized educational measurement practices. Namely, an inherent contradiction between guidelines for how scales/tests are developed and how those scales/tests are typically analyzed. Best practices for developing scales/tests emphasize developing individual items or subsets of items to capture unique aspects of constructs, such that the full construct is captured across the test. Analysis approaches, typically factor analysis or related reflective models, assume that no individual item (nor subset of items) captures unique, construct-relevant variation. This contradiction has important implications for the use of factor analysis to support measurement claims. The implications and other critiques of factor analysis are discussed.