What does this research mean for the field?

Using inferential statistics to evaluate the quality of group matching is theoretically and practically problematic, and matching quality should instead be assessed through descriptive comparisons informed by expert knowledge. Novelty: ClaimNovelty.METHODOLOGICAL. Consensus alignment: ConsensusAlignment.CHALLENGES_CONSENSUS.

What question did this study set out to answer?

To examine theoretical and practical shortcomings of applying inferential statistics to assess group matching on confounding variables.

August 23, 2026International Journal of BilingualismOpen Access

Do Not Use Inferential Statistics to Evaluate Quality of Matching

View Full Paper

Ask AI

Bookmark

Authors

SFStefan L. Frank

Discussion

Member takes

Overview

Methodological analysis reveals flaws in using inferential statistics to evaluate group matching in bilingualism studies, suggesting researchers rely on descriptive distributions and domain expertise.

Key Points

To examine theoretical and practical shortcomings of applying inferential statistics to assess group matching on confounding variables.
Simulated fictitious datasets to illustrate the statistical failure modes of inferential tests (such as t-tests) when evaluating matching quality.
Evaluated group matching reporting practices across articles from two bilingualism journals using logistic regression.
Demonstrates that inferential statistical tests fail to validate whether groups are adequately matched on confounding variables.
Concludes that quantitative cutoffs cannot substitute for qualitative evaluations of full sample distributions supported by domain-specific expertise.

Cite This Study

Stefan L. Frank (2026) studied this question.

synapsesocial.com/papers/6a8aadb47677a3411444610b https://doi.org/10.1177/13670069261475930

View Full Paper

Ask AI

Bookmark

August 23, 2026International Journal of BilingualismOpen Access

Do Not Use Inferential Statistics to Evaluate Quality of Matching

View Full Paper

Ask AI

Bookmark

Authors

SFStefan L. Frank

Discussion

Member takes

Overview

Key Points

To examine theoretical and practical shortcomings of applying inferential statistics to assess group matching on confounding variables.
Simulated fictitious datasets to illustrate the statistical failure modes of inferential tests (such as t-tests) when evaluating matching quality.
Evaluated group matching reporting practices across articles from two bilingualism journals using logistic regression.
Demonstrates that inferential statistical tests fail to validate whether groups are adequately matched on confounding variables.
Concludes that quantitative cutoffs cannot substitute for qualitative evaluations of full sample distributions supported by domain-specific expertise.

Cite This Study

Stefan L. Frank (2026) studied this question.

synapsesocial.com/papers/6a8aadb47677a3411444610b https://doi.org/10.1177/13670069261475930

View Full Paper

Ask AI

Bookmark