On the validity of fMRI studies with subject-level data processed through different pipelines

In recent years, the lack of reproducibility of research findings has become an important source of concerns in many scientific fields, including functional Magnetic Resonance Imaging (fMRI). The low statistical power often observed in fMRI studies was identified as one of the leading causes of irreproducibility. The development of data sharing opens up new opportunities to achieve larger sample sizes by reusing existing data. FMRI studies proceed by first preparing subject-level data using a given analysis pipeline and then combining those into a group analysis. Historically the subject-level analysis pipeline was identical for all subjects. As practices evolve towards more data reuse, researchers might want to directly combine subject-level data thata were processed using different pipelines. Here, we investigate the impact of combining subject-level data processed with different pipelines in between-group fMRI studies. We used the HCP Young-Adult dataset (N=1,080 participants) processed with 24 different pipelines. We then performed between-group analyses comparing subject data processed with different pipelines. We worked under the null hypothesis of no differences between subjects and compared the estimated false-positive rates obtained with the nominal rates. We showed that the analytical variability induced by the parameters explored in this dataset increases the false positive rates of studies combining data from different pipelines. We conclude that different processed subject data cannot be combined without taking into account the processing applied on these data.