Methodological quality of systematic reviews on medical education. A meta‐research

To assess the characteristics and methodological quality of systematic reviews on medical education. This meta‐research considered systematic reviews on medical education published in MEDLINE (PubMed). The following outcomes were assessed: (i) general characteristics of systematic reviews, (ii) frequency and type of methodological tool and/or guidance used for conducting the systematic review, (iii) adequacy to the AMSTAR‐2 items, (iv) association between the methodological tool/guide used and the adequacy of the AMSTAR‐2 and (v) frequency of systematic reviews that referred to the use of the PRISMA statement for their reporting. Forty‐eight systematic reviews published between 2004 and 2023 were included; 77.1% addressed intervention questions, and the outcomes assessed were predominantly knowledge, attitude, skills and behaviour. According to the AMSTAR‐2 assessment, 93.8% of the systematic reviews were classified as critically low quality, including 91.9% of intervention reviews and 100% of reviews addressing frequency or unclear review questions. Overall, 43.8% inappropriately reported using the PRISMA statement as methodological guidance, whereas 18.8% reported using the PRISMA statement as a reporting guideline. Eight reviews (16.7%) followed the methodological recommendations of the Cochrane Handbook or BEME, and three of them were classified as low or moderate quality. Reviews that reported using an appropriate methodological guide showed higher AMSTAR‐2 adequacy (63% [47–74] versus 35% [27–42], p = 0.0016). Considering the substantial methodological limitations identified across the included systematic reviews, there is an evident need for guidance and training on the methodological process of systematic reviews in medical education, including the planning, conduct and reporting stages, to improve the quality of the evidence supporting teaching and learning strategies. Context and implications Rationale for this study: Systematic reviews in medical education have increased substantially, yet their methodological rigour and adherence to established standards remain uncertain. Why the new findings matter: This meta‐research found that most systematic reviews in medical education are of critically low methodological quality and frequently misuse reporting guidelines, such as PRISMA, as methodological frameworks. Implications for researchers, educators and journal editors: Researchers would benefit from targeted training on the appropriate planning and conduct of systematic reviews, using established methodological and reporting frameworks (e.g. Cochrane, BEME, PRISMA). Educators and editors should encourage methodological literacy, promote adherence to standards and strengthen peer‐review processes to improve the reliability and usefulness of evidence supporting teaching and learning in medical education.