What does this research mean for the field?

Default settings and undocumented implementation choices across major statistical software packages can alter results and flip statistical significance in boundary cases, undermining research reproducibility. Novelty: ClaimNovelty.CONTRADICTORY. Consensus alignment: ConsensusAlignment.CHALLENGES_CONSENSUS.

What question did this study set out to answer?

To identify, catalog, and evaluate undocumented implementation decisions and default settings in major statistical packages that lead to conflicting statistical conclusions on identical data.

September 5, 2026The American Statistician

Hidden Decisions in Statistical Software: How Default Settings and Implementation Choices Undermine Reproducibility

View Full Paper

Ask AI

Bookmark

Authors

YWYang Wang

Discussion

Member takes

Overview

Comparative software analysis reveals that default settings alter significance verdicts across five statistical packages, indicating an overlooked computational cause of the reproducibility crisis.

Key Points

To identify, catalog, and evaluate undocumented implementation decisions and default settings in major statistical packages that lead to conflicting statistical conclusions on identical data.
Evaluated 37 hidden implementation choices across five statistical software packages (SPSS, R, SAS, Stata, and Python statsmodels) across five common statistical tests.
Tested software behavior using controlled boundary datasets where reference p-values were near 0.05, tracing discrepancies directly to package documentation and source code.
Demonstrated that default software implementation choices can flip qualitative statistical significance verdicts from significant to nonsignificant on boundary datasets.
Identified five high-impact sources of divergence: Poisson GLM dispersion handling, Cox regression tie-handling (Breslow versus Efron), standard error computation for noncanonical GLMs, finite-sample degrees-of-freedom versus Wald-z inference in mixed models, and mixed model boundary-fit diagnostics.

Cite This Study

Yang Wang (2026) studied this question.

synapsesocial.com/papers/6a9bd4436b95aff0620ebc6d https://doi.org/10.1080/00031305.2026.2729003

View Full Paper

Ask AI

Bookmark

September 5, 2026The American Statistician

Hidden Decisions in Statistical Software: How Default Settings and Implementation Choices Undermine Reproducibility

View Full Paper

Ask AI

Bookmark

Authors

YWYang Wang

Discussion

Member takes

Overview

Comparative software analysis reveals that default settings alter significance verdicts across five statistical packages, indicating an overlooked computational cause of the reproducibility crisis.

Key Points

To identify, catalog, and evaluate undocumented implementation decisions and default settings in major statistical packages that lead to conflicting statistical conclusions on identical data.
Evaluated 37 hidden implementation choices across five statistical software packages (SPSS, R, SAS, Stata, and Python statsmodels) across five common statistical tests.
Tested software behavior using controlled boundary datasets where reference p-values were near 0.05, tracing discrepancies directly to package documentation and source code.
Demonstrated that default software implementation choices can flip qualitative statistical significance verdicts from significant to nonsignificant on boundary datasets.
Identified five high-impact sources of divergence: Poisson GLM dispersion handling, Cox regression tie-handling (Breslow versus Efron), standard error computation for noncanonical GLMs, finite-sample degrees-of-freedom versus Wald-z inference in mixed models, and mixed model boundary-fit diagnostics.

Cite This Study

Yang Wang (2026) studied this question.

synapsesocial.com/papers/6a9bd4436b95aff0620ebc6d https://doi.org/10.1080/00031305.2026.2729003

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

View Full Paper

Ask AI

Bookmark