Preregistration of hypotheses and analysis pipelines in EEG/ERP research is proposed as a solution to limit confirmation and hindsight bias caused by undisclosed analytic flexibility.
A combination of confirmation bias, hindsight bias, and pressure to publish may prompt the (unconscious) exploration of various methodological options and reporting only the ones that lead to a (statistically) significant outcome. This undisclosed analytic flexibility is particularly relevant in EEG research, where a myriad of preprocessing and analysis pipelines can be used to extract information from complex multidimensional data. One solution to limit confirmation and hindsight bias by disclosing analytic choices is preregistration : researchers write a time-stamped, publicly accessible research plan with hypotheses, data collection plan, and intended preprocessing and statistical analyses before the start of a research project. In this manuscript, we present an overview of the problems associated with undisclosed analytic flexibility, discuss why and how EEG researchers would benefit from adopting preregistration, provide guidelines and examples on how to preregister data preprocessing and analysis steps in typical ERP studies, and conclude by discussing possibilities and limitations of this open science practice.
Paul et al. (Wed,) conducted a review in EEG/ERP research methodology. Preregistration was evaluated. Preregistration of hypotheses and analysis pipelines in EEG/ERP research is proposed as a solution to limit confirmation and hindsight bias caused by undisclosed analytic flexibility.