Key result
Routine confidence intervals can combat the scientific decline effect by exposing unreliable or missed effects.
Routine use of confidence intervals, adjusted for multiple comparisons, can help combat the scientific decline effect by providing bounds on true effect sizes.
Answers are grounded in this paper and cite their sources.
May warrant CI reporting in CV trials to bound effects; leaves open reversal of decline effect without broader reforms.
A symptom of the need for greater reproducibility in scientific practice is the decline effect-the fact that the size of many experimental effects decline with subsequent study or fail to replicate entirely. A simple way to combat this problem is for scientists to more routinely use confidence intervals (CIs) in their work. CIs provide frequentist bounds on the true size of an effect and can reveal when a statistically significant effect is possibly too small to be reliable or when a large effect might have been missed due to insufficient statistical power. CIs are often lacking in psychophysiological reports, likely due to the large number of dependent variables, which complicates deriving and visualizing CIs. In this article, I explain the value of CIs and show how to compute them for analyses involving multiple variables in various ways that adjust the intervals for the greater uncertainty induced by multiple statistical comparisons. The methods are illustrated using a basic visual oddball ERP dataset and freely available MATLAB software.
No takes yet. Share an insight, caveat, or question.
David M. Groppe (2016) studied this question. Confidence intervals was evaluated. Routine use of confidence intervals can combat the scientific decline effect by revealing when significant effects are too small to be reliable or when large effects are missed.
Synapse has enriched 10 related papers. Consider them for comparative context: