Rating Down for Publication Bias is Rare in Cochrane Reviews: A Descriptive Study

Background: Publication bias impacts the direction and magnitude of summary effect estimates in systematic reviews and the confidence in the review findings. It remains unclear to what degree publication bias is considered in the certainty assessments in systematic reviews. Our research aims are to generate an overview of and estimate how frequent rating down occurs for the GRADE domain "publication bias" across Cochrane reviews. Study Design and Setting: This is a descriptive study. We included Cochrane reviews published between January 2021 and October 2023 that included at least one outcome for which the certainty of evidence was rated down for publication bias. We collected data from the reviews' full text, analyses, and figures. We analyzed data with descriptive statistical analysis. We used a deductive qualitative approach for analyzing qualitative data. Results: We identified 81 reviews. These reviews included between 1 and 328 individual studies. All but three reviews included randomized-controlled trials only. Data from less than 10 studies supported the summary of findings of 106 outcomes. The most commonly used methods for detecting publication bias were funnel plots. 220 outcomes were rated down by one level for publication bias. Four outcomes were rated down by two levels. The most commonly reported reasons were indications of publication bias in funnel plots. Conclusions: Rating down for publication bias was rare in Cochrane reviews despite the high prevalence of publication bias in original research. Guidance might not suffice to inform assessments of publication bias in reviews with less than 10 studies.