Answers are grounded in this paper and cite their sources.
The .05 boundary within Null Hypothesis Statistical Testing (NHST) “has made a lot of people very angry and been widely regarded as a bad move” (to quote Douglas Adams). Here, we move past meta-scientific arguments and ask an empirical question: What is the psychological standing of the .05 boundary for statistical significance? We find that graduate students in the psychological sciences show a boundary effect when relating p -values across .05. We propose this psychological boundary is learned through statistical training in NHST and reading a scientific literature replete with “statistical significance” . Consistent with this proposal, undergraduates do not show the same sensitivity to the .05 boundary. Additionally, the size of a graduate student’s boundary effect is not associated with their explicit endorsement of questionable research practices. These findings suggest that training creates distortions in initial processing of p -values, but these might be dampened through scientific processes operating over longer timescales.
No takes yet. Share an insight, caveat, or question.
Rao et al. (2024) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: