Psychometric Reliability of ERN and Pe Across Flanker, Stroop, and Go/No-Go Tasks: A Direct and Conceptual Replication

The effectiveness of error-related negativity (ERN) in assessing individual differences hinges on its internal consistency. Despite evidence that task used to record ERN moderates internal consistency, this moderation is rarely examined within the same sample, risking inaccurate generalizations of psychometrics. A direct and conceptual replication of Meyer et al. (2013, Psychophysiology) was conducted in 182 participants to assess internal consistency of ERN from flanker, go/no-go, and Stroop tasks as a function of increasing trials. Analyses were extended to include error positivity (Pe) and difference scores (Delta-ERN, Delta-Pe; error minus correct), and generalizability theory and multilevel models were used to statistically compare internal consistency across tasks. Findings replicated the internal consistency of ERN scores for a go/no-go task but not for flanker and Stroop tasks. ERN scores from a flanker task showed lower internal consistency than previously reported, whereas those from a Stroop task were higher. Although data from a go/no-go task showed the highest internal consistency with the fewest trials, a flanker task showed higher internal consistency of ERN and Delta-Pe than go/no-go and Stroop tasks and higher internal consistency of Delta-ERN than a go/no-go task when examining all recorded error trials. The results emphasize the critical need to evaluate and report internal consistency on a study-by-study basis and caution against using universal trial cutoffs for data inclusion. Moreover, the exclusive use of a single task is discouraged due to the ongoing need to understand the functional significance of ERN and Pe and their moderation by task effects.