Confirmation Bias Without Prior Beliefs: Should We Speak of a ‘See-Something-Versus-Nothing Bias’

Significant effects still receive more attention in research and are published more often than null effects. This contributes to a considerable publication bias across disciplines. In this registered report, I examine the question of whether, in addition to motivational processes, perceptual processes can also be held responsible for an overestimation of significant versus insignificant effects. To test this assumption, 4 studies are planned in which participants will evaluate 30 scientific abstracts in a fictitious research field across which the proportion of significant versus null effects will be experimentally varied (10, 30, 50, 70, 90 %) and whether the research topic is disclosed or masked. It will be tested to what extent different groups (Study 1: laypeople, Study 2: students, Study 3: researchers) will overestimate the quantity and quality of significant compared to null effects. Study 4 will replicate this design in a heterogeneous sample, but it will be additionally tested whether prior information letters (versus no information) may help to overcome the overestimation of significant effects. Overall, the studies aim to provide a broader picture of the misalignment in the publication system and to suggest ways to make science more transparent.