The Different Meanings of Response Variability: A Preregistered Replication and Extension

Response variability in questionnaires may reflect substantive differences, stable response tendencies, as well as careless responding, yet these interpretations are rarely examined together. In a preregistered laboratory study, 67 participants completed the same questionnaire under two truthful, an overtly careless, and a covertly careless instruction while adherence was monitored. We examined whether corrected response variability generalized across four content-diverse measures and how well common careless-responding indicators distinguished truthful from careless records. For the preregistered item-set operationalization, cross-measure correlations averaged r = .40 and .45 across the two truthful conditions, whereas the corresponding strict Baird-style correlations averaged r = .20 and .26. Overall corrected variability was highly stable across truthful assessments (item-set r = .92; strict r = .83) and remained moderately stable between truthful and careless conditions. Overall raw item-set variability was lower under careless instructions, whereas strict Baird-style variability was highest under covert carelessness. For carelessness in adherence-confirmed analyses, page time showed the strongest discrimination (AUCs = .85~.88), followed by retest correspondence, attention checks, and fake-adjective accuracy; longstring performed weakly. These findings indicate that response variability contains stable individual differences while also being sensitive to task engagement.