Establishing the psychometric properties of three measures of speech perception

There is a renewed interest in explaining individual variation across the cognitive sciences, including in the domain of psycholinguistics. A major barrier towards this goal is the unknown psychometric stability of common speech perception tasks—whether performance in these tasks reflect stable traits or temporary states. This study evaluated content validity and test–retest reliability for three widely used speech perception tasks: two-alternative forced-choice (2AFC) identification, AX discrimination, and visual analogue scale (VAS) identification. In each of two sessions separated by 1 week in time, participants (n = 300) completed a single task for two stimulus sets. The phonetic property of speech (voice-onset-time) was constant across stimulus sets though the talker and word differed. Individual performance was quantified using many task-specific metrics (e.g., 2AFC boundary, AX sensitivity, VAS slope, VAS consistency). Despite remarkable group-level consistency, content validity was poor for all individual difference measures. Test–retest reliability was generally better but rarely met threshold for psychometric stability (defined as r = 0.70). These findings suggest that individuals’ performance on a given stimulus may not reliably index stable traits that generalize across stimuli or time points, even when the phonetic property is held constant, which poses significant constraints on using these tasks for individual differences research.