PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
May 16, 2026Educational Psychology Review0 citationsOpen Access

From RAMSing to Rigor: Improving Moderator Analysis in STEM Education Meta-Research

MBMehmet BıçakçıFHFabian HellerHSHeidrun Stoeger

Key Points

  • This research aims to explore the effects of moderator analysis in STEM education meta-research, identifying biases and proposing framework improvements.
  • Conducted an umbrella review of 89 meta-analyses with 1,786 moderator effect sizes.
  • Utilized two Bayesian frameworks, RoBMA-PSMA and RoBMA-Regression, for bias adjustment.
  • Performed prospective power analyses to determine necessary effect sizes for accurate moderator testing.
  • Found that non-significant moderator effect sizes had notably lower visibility than significant ones (ω ≈ 0.40).
  • The overall bias-adjusted mean moderator effect was g = 0.27 (95% CrI [0.20, 0.34]) with substantial heterogeneity (τ = 0.48).
  • Highlighted the need for approximately 14 independent effect sizes per moderator level to reliably detect typical moderator effects.

Abstract

Abstract Moderators are intended to clarify when, where, and for whom interventions work, yet null moderator results are often omitted from the published record. We term this practice “reporting after moderators are significant” (RAMSing) and examine its scope and consequences in STEM education meta-research. Drawing on an umbrella review of 89 meta-analyses comprising 1, 786 moderator effect sizes, we found a substantial visibility gap: non-significant moderator effect sizes showed markedly lower relative visibility than the most strongly selected results (\: \: \: \: 0. 40). To address this bias, we combined structural mapping with two bias-robust Bayesian frameworks, RoBMA-PSMA and RoBMA-Regression, to estimate bias-adjusted moderator effects, derive field-specific benchmarks, and estimate realistic evidential requirements for moderator testing. The overall bias-adjusted mean moderator effect was g = 0. 27 (95% CrI 0. 20, 0. 34) with substantial heterogeneity (\: \: = 0. 48). Modeling a five-level moderator-theme classification substantially improved fit (BF Inc = 350) and slightly reduced residual heterogeneity (\: \: = 0. 44). RAMSing-adjusted benchmarks placed the 25th, 50th, and 75th percentiles of | g | at 0. 18, 0. 38, and 0. 64, respectively, all below both Cohen’s conventional heuristics and the raw empirical quartiles. Prospective power analyses further indicated that, under the assumptions examined here (g = 0. 38, \: {\: }^2 = 0. 24, \: \: = 0. 05), approximately 14 independent effect sizes per moderator level are needed to detect a typical moderator effect. These findings indicate that RAMSing is likely an important source of distortion in moderator-based inference within this corpus. The workflow and openly available toolkit introduced here provide a practical framework for detecting and bias-adjusting RAMSing, and for supporting practices designed to reduce it in STEM education and other fields that rely on meta-analytic moderator evidence.

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

Bıçakçı et al. (2026) studied this question.

synapsesocial.com/papers/6a080b4ea487c87a6a40d74chttps://doi.org/10.1007/s10648-026-10172-1
Ask AI
Helpful
Bookmark
Share
View Full Paper