Answers are grounded in this paper and cite their sources.
There is often a bias towards internal validity when analyzing the methodological quality of interventions. In this study, we examined the convergent and discriminant validity evidence and reliability of the Methodological Quality Scale (MQS), a tool aimed at assessing the degree to which researchers implement measures to avoid bias in their experimental and quasi-experimental studies, considering internal, external, and construct validity. Fifty-one research papers were randomly selected and two previously trained independent coders used the MQS, the Downs & Black (D&B), the Physiotherapy Evidence Database (PEDro), and the Risk of Bias 2 (RoB-2) scales to code them. Reliability was evaluated using McDonald’s ω, Cronbach’s α, the unbiased estimator of Cronbach’s α ( \:-\: ), and Guttman’s λ6, and descriptive statistics for the subscales were reported. The multitrait-multimethod matrix with Spearman’s correlations (ρ) was used to examine the relationship between subscales, i.e., convergent and discriminant validity evidence. The results yielded adequate reliability across all MQS dimensions. Convergent validity was evidenced by moderate correlations in the same dimension of the MQS with established instruments (e.g., internal validity with PEDro). Discriminant validity was supported by the absence of significant correlations between different dimensions (e.g., between MQS Internal Validity and D&B External Validity, and MQS Construct Validity with D&B and PEDro External Validity). These findings support the use of the MQS, a 10-item tool that assesses internal, external and construct dimensions, especially for educational and psychological interventions where randomized controlled trials may not always be feasible.
No takes yet. Share an insight, caveat, or question.
A 2026 study studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: