An Empirical Evaluation of Factor Reliability

The psychometric reliability of a factor, defined as its generalizability across samples drawn from the same population of tests, is considered as a necessary precondition for the scientific meaningfulness of factor analytic results. A solution to the problem of generalizability is illustrated empirically on data from a set of tests designed to measure facets of response styles and of personality dimensions. Parallel sets of measures based on personality scales defining each of seven factors were separakely factored. Independent sets of component scores derived from the orthogonal least squares fit to the oblique factor pattern matrix were computed, and these component scores were intercorrelated between the two sets, yielding factor reliabilities, whose values ranged from .66 to .86 (p < .0001, for each factor). A corresponding analysis based on scores derived from random binary data yielded nonsignificant factor reliabilities ranging from -.12 to +.07. It was recommended that such a test of factor generalizability be incorporated routinely into fador analytic investigations, particularly those employing Procrustes-type rotations.