Transitivity in Pairwise Comparisons and the Heterogeneity of Psychological Attributes

Likert scales rely on the assumption that psychological attributes are ordinal, meaning that the attribute levels a, b, and c are in transitive relations. Testing transitivity is possible through pairwise comparisons of attribute levels. Previous tests showed that people can violate transitivity, which calls into question the basic assumption of rating scales. We hypothesized that transitivity violations might be related to the large number of pairwise comparisons that respondents made. We showed that a reduced number of comparisons per respondent was associated with a small number of violations, although about a quarter of respondents still violated transitivity. We explained this by the heterogeneity of psychological attributes. We discussed intransitivity in heterogeneous systems, and the importance of studying it in psychology.