Replicating the ‘Seductive Allure of Neuroscience Explanations’ effect in a classroom experiment and an online study

The “Seductive Allure of Neuroscience Explanations” (SANE) effect refers to the observation that superfluous neuroscience information (SNI) added to an explanation can bias judgments of information quality. This and other related findings have called for education measures that enhance the ability to distinguish good explanations from bad ones and relevant information from irrelevant one. We here report the results of a classroom experiment to sensitize undergraduate students to that issue. The experiment built on the seminal study by Weisberg et al. (2008) on the SANE effect. Over the years, the experiment revealed considerable variation between student cohorts, highlighting the problem of small sample sizes, a finding that became itself a topic in the classroom. Inspired by these observations, we then set out to conceptually replicate the original study using an online experiment that – extending the original study into the German-speaking community – allowed us to directly assess the statistical interactions between explanation quality, the presence of SNI and expertise levels in German and international participants. In this preregistered study, participants (n=409), spanning six different levels of expertise, rated the quality of good and bad explanations, with or without SNI. Irrespective of the presence of SNI, participants across all expertise levels rated good explanations more favorably than bad ones (p=<0.001), but the differences were surprisingly small, and even among highly educated participants, bad explanations were rated favorably. We also found a significant two-way interaction between the impact of SNI and expertise, with SNI boosting ratings mostly in participants with less expertise (p=<0.001). There was no significant effect in relation to the language used. Developing a curriculum that trains students to distinguish between actual explanations and “bullshit” would ultimately also sensitize teachers and experts that produce and judge such language. Such training measures should also play a role in science communication more broadly.