A Reliability Generalization Meta-Analysis of the Mathematics Self-Efficacy

Mathematical self-efficacy is an individual's belief in their ability to solve mathematical problems, understand mathematical concepts, and successfully complete mathematical tasks. This study uses reliability data reported in various studies that employed the “Mathematics Self-Efficacy Scale” (MSES) developed by Umay (2001) to determine individuals' levels of self-efficacy in mathematics. It aims to estimate the average (combined) reliability coefficient using this data. This study was conducted to estimate the average (combined) reliability coefficient using the reliability data reported in different studies that used this scale and to examine the reasons for the variation in the reliability coefficients reported in these studies. In this study, which employed reliability generalization, one of the meta-analysis methods, analyses were conducted by combining the transformed coefficient values obtained via the Bonett transformation from 32 Cronbach’s alpha coefficients derived from 31 independent studies with a total sample size of 10,072, using a random effects model. Publication bias analyses have shown that there were no statistically significant publication bias findings in this study. The average Cronbach's alpha value of the MSES was estimated to be .829 (95% CI [.812 - .844]), and this estimated value was found to be statistically significant (p <.01). Analyses conducted on heterogeneity determined that there was a high level of heterogeneity among the Cronbach's alpha coefficients obtained from the studies (I² = %88,98). Moderator analyses have shown that potential variables such as the sample group, publication status, research design, region, year of publication, sample size, percentage of men, and mean scale score do not significantly explain this observed heterogeneity. The substantial heterogeneity identified (I² = 88.72%) constitutes the central finding of this study, demonstrating that MSES reliability is not transportable across studies or populations. The 95% prediction interval (PI [.726, .893]) makes this practically concrete: a researcher using the MSES in a new study should expect an alpha value anywhere in this range, not necessarily near the pooled mean of .829. These findings carry direct implications for reliability reporting norms: journal editors, peer reviewers, and researchers using the MSES or similar instruments should treat sample-specific reliability estimation as a non-negotiable methodological standard rather than an optional practice.