Answers are grounded in this paper and cite their sources.
Background Most scientific studies report inferential statistical results using p -values. For clinical decision-making, correctly interpreting the results of significance tests is particularly important, for example, to differentiate between statistical significance and clinical relevance. However, p -values are frequently misinterpreted, and reporting effect sizes alongside p -values has therefore been recommended to support the interpretation of statistical results. Methods We examined 121 medical students’ misconceptions about p -values when presented with three fictitious but realistic medical abstracts containing comparisons of group means and t -tests. Medical context, sample size, p -value, and effect size were systematically varied across the abstracts (abstract 1: N 1 = N 2 = 7,500; p < 0.001; Cohen’s d = 0.12; abstract 2: N 1 = N 2 = 10; p = 0.04; Cohen’s d = 1.00; abstract 3: N 1 = N 2 = 40; p = 0.09; Cohen’s d = 0.38). We varied the amount of statistical information provided in each abstract (information set 1: sample size and p -value; information set 2: sample size, p -value, means, and standard deviations; information set 3: the same information and effect size Cohen’s d ). We examined whether additional statistical information was associated with fewer misconceptions and how the provided information influenced students’ perceived effect size. Results In all versions, a high proportion of medical students across all semesters of study demonstrated multiple misconceptions about p -values. The number of endorsed misconceptions substantially decreased with the provision of more statistical information only in the context with a large sample size, a statistically significant result, and a very small effect size. In this context, providing additional statistical information reduced the tendency to infer clinical relevance and effect magnitude directly from statistical significance. Conclusions Although reporting standardized effect sizes has repeatedly been recommended, their inclusion alone appears insufficient to prevent mis- and overinterpretations of inferential statistical results. The interpretation of statistical evidence should be addressed more intensively and longitudinally throughout medical education, including by explicitly addressing typical misconceptions. Alternative teaching approaches to statistical inference, including simulation-based approaches, should also be considered.
No takes yet. Share an insight, caveat, or question.
Eisenhut et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: