Inferences of p-values, statistically significant results, and confidence intervals – A systematic review of empirical studies

This systematic review aimed to investigate how researchers, educators, students, and professionals understand and interpret statistical inference. Following PRISMA guidelines, four databases were searched through 8 August 2025 with no time restrictions for empirical studies on interpretations of p-values, statistical significance, and confidence intervals. Fifty-five studies published between 1963 and 2024 were included, encompassing over 15,000 participants from diverse regions, including Europe, North America, Australia, China, South America, and the Middle East. Studies primarily used written or online cross-sectional surveys, employing various single questions or questionnaires to assess understanding and inferential reasoning about statistical results. Participants ranged from students and PhD candidates to researchers, clinicians, statisticians, and methodology instructors with substantial statistical education and experience. Results showed widespread and persistent misunderstandings across all groups. Common errors about p-values and null hypothesis testing included the inverse probability fallacy, misinterpreting p-values as probabilities of hypotheses, the replication fallacy, and conflating statistical significance with clinical or practical relevance. Confidence intervals were often misinterpreted as probabilistic ranges for true parameters or as predictors of future outcomes. Presenting effect estimates with confidence intervals rather than p-values reduced some misinterpretations, but high error rates persisted. These findings highlight the need for improved statistical education, clearer communication of core concepts, and careful presentation of results in scientific literature. Misinterpretations of statistical inference are pervasive, with implications for research reproducibility, scientific literacy, and evidence-based decision-making.