Answers are grounded in this paper and cite their sources.
Theoretical analysis reveals misalignments between scientific goals and multiple testing adjustments in research communities, highlighting conditions where multiplicity corrections should be omitted.
When many statistical hypotheses are evaluated simultaneously, statisticians often recommend adjusting (or “correcting”) standard hypothesis tests. In this article, I (1) distinguish two senses of adjustment, (2) investigate the prudential and epistemic goals that adjustment might achieve, and (3) identify conditions under which a researcher should not adjust for multiplicity in the two senses I identify. I tentatively conclude that the goals of scientists and the public may be misaligned with the decision criteria used to evaluate multiple-testing regimes.
No takes yet. Share an insight, caveat, or question.
Conor Mayo‐Wilson (2024) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: