Assessment of the Multitrait-Multimethod Matrix Validity of Likert Scales Via Confirmatory Factor Analysis

Two samples of adults were administered nine attitude scales: three constructs (attitudes toward discipline of children, mathematics, and the law) each measured by three methods (Likert, Thurstone and Semantic Differential techniques). For one sample a dichotomous Likert response format was used, in the other a nine-point Likert continuum. Each of the two resulting 9 x 9 MTMM matrices was assessed for evidence of validity using confirmatory factor analysis, supplemented by an estimation of variance components analysis, and the results compared across matrices. The results showed that the three methods converged across the three constructs but that there was an interaction effect such that it did make a difference which method was chosen to measure any particular construct. There was more shared variance among variables in the matrix where the nine-point Likert scales had been used. The results indicated (a) greater validity of Likert scales with the use of a larger number of scale steps, in the sense that total scores more highly estimated the construct underlying the item set, and (b) the value of conceptualizing "types" of measurement techniques which are nevertheless not interchangeable in any particular context.