Answers are grounded in this paper and cite their sources.
Methodological analysis identifies low quality in systematic reviews and protocol registrations of observational studies, suggesting improvements are needed.
OBJECTIVE: The objective of this study was to determine the methodological quality of published systematic reviews of observational studies and their respective protocol registrations. INTRODUCTION: Historically, methodological guidance for systematic reviews has focused on systematic reviews of randomized controlled trials. Guidance for systematic reviews of observational studies emerged about a decade ago. However, little is known about the methodological quality of systematic reviews of observational studies published to date. ELIGIBILITY CRITERIA: Systematic reviews were eligible for inclusion if they i) included observational studies and quasi-experimental studies with human participants, ii) extracted exposure-outcome effect estimates from primary studies, iii) were written in English, and iv) were published in a Q1 Web of Science-indexed or Scopus-indexed epidemiology journal between January 1, 2023, and December 31, 2023 (including advance online publications). METHODS: This methodological study is a follow-up analysis of a random sample of 50 published systematic reviews of observational epidemiologic studies that were originally identified to examine whether control for third variables was considered as a source of bias or heterogeneity. Where possible, the methods were informed by JBI guidance for umbrella reviews. English-language Q1 epidemiology journals in Scopus and Web of Science were searched to identify records. The search was first performed on January 16, 2024, and repeated on April 1, 2024. For this follow-up analysis, protocol registrations corresponding to the 50 included systematic reviews were retrieved. Data extraction and critical appraisal of the full-text systematic reviews and the protocol registrations were performed independently by 2 reviewers. Methodological quality of the full-text systematic reviews and protocol registrations was assessed using items 1-9 from the JBI Critical Appraisal Checklist for Systematic Reviews and Research Syntheses. RESULTS: Of the 50 included systematic reviews, 25 had protocol registrations. The median number of methodological quality items met by the full-text systematic reviews with a protocol registration was 3 (range 1-7) and without a protocol registration was 2 (range 0-5). Item 5, concerning the conduct of critical appraisal, was the only item on which systematic reviews with a protocol registration scored statistically significantly better than systematic reviews without a protocol registration. The median number of methodological quality items met by the protocol registrations was 3 (range 1-6). Agreement in the methodological quality assessment between full-text systematic reviews and protocol registrations was generally high (ie, items were often met or not met across both full-text systematic reviews and protocol registrations). CONCLUSIONS: The methodological quality of the included systematic reviews of observational studies and their respective protocol registrations was generally low. Further research is needed to identify effective strategies for improving the methodological quality of systematic reviews of observational studies.
No takes yet. Share an insight, caveat, or question.
Rijnhart et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: