Answers are grounded in this paper and cite their sources.
Vul, Harris, Winkielman, and Pashler (2009, this issue) claim that many brain-personality correlations in fMRI studies are ''likely . . . spurious'' (p. 274), and ''should not be believed'' (p. 285). Several of their con- clusions are incorrect. First, they incorrectly claim that whole-brain regressions use an invalid and ''nonindepen- dent'' two-step inferential procedure, a determination based on a survey sent to researchers that only included nondiagnostic questions about the descriptive process of plotting one's data. We explain how whole-brain regres- sions are a valid single-step method of identifying brain regions that have reliable correlations with individual difference measures. Second, they claim that large corre- lations from whole-brain regression analyses may be the result of noise alone. We provide a simulation to demon- strate that typical fMRI sample sizes will only rarely produce large correlations in the absence of any true ef- fect. Third, they claim that the reported correlations are inflated to the point of being ''implausibly high.'' Though biased post hoc correlation estimates are a well-known consequence of conducting multiple tests, Vul et al. make inaccurate assumptions when estimating the theoretical ceiling of such correlations. Moreover, their own ''meta- analysis'' suggests that the magnitude of the bias is ap- proximately.12—a rather modest bias. In an article in this issue, Vul, Harris, Winkielman, & Pashler (2009, this issue) claim that brain-personality correlations in many social neuroscience studies and those in related fields are ''implausibly high'' (p. 274), ''likely . . . spurious'' (p. 274), and ''shouldnotbebelieved''(p.285).Thearticlewasoriginallytitled ''VoodooCorrelationsinSocialNeuroscience''andwascirculated widely in the scientific community, on the Internet, and in the popularpresspriortopublication.Thewordvoodoo,asappliedto science, carries a strong and specific connotation of fraudulence, aspopularizedbyRobertPark's(2000)book,VoodooScience:The Road From Foolishness to Fraud. Though the title was subse- quently changed to remove the wordvoodoo, the substance of the article and its connotations are unchanged: It is a pointed attack on social neuroscience. Much of the article's prepublication im-
No takes yet. Share an insight, caveat, or question.
Lieberman et al. (2009) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: