Assessment of the Methodological Quality of Systematic Reviews Published in the Urological Literature From 1998 to 2008

PURPOSE: Well done systematic reviews provide the highest quality evidence for clinical questions of therapeutic effectiveness. We assessed the methodological quality of systematic reviews in the urological literature. MATERIALS AND METHODS: We systematically investigated all systematic reviews published in 4 major urological journals from 1998 to 2008. Studies were identified using a predefined search strategy in PubMed and confirmed by a hand search of journal tables of contents. A validated 11-point instrument to assess the methodological quality of systematic reviews was applied by 2 independent reviewers after a pilot testing phase. Disagreements were discussed and resolved by consensus. RESULTS: The systematic literature search identified 217 individual systematic reviews, of which 57 ultimately met study eligibility criteria. Ten (17.5%), 20 (35.1%) and 27 (47.4%) systematic reviews were published in 1998 to 2001, 2002 to 2005 and 2006 to 2008, respectively. Using the measurement tool to assess systematic reviews the mean +/- SD score was 4.8 +/- 2.0 points. Fewer than half of all systematic reviews performed a systematic literature search that included at least 2 databases (49.1%) or unpublished studies (31.6%), or provided a list of included and excluded studies (45.6%). Of the systematic reviews 63.2% assessed and documented the methodological quality of included studies. Systematic reviews with The Cochrane Collaboration authorship affiliation had a higher mean score than those with no such reported affiliation (6.5 +/- 1.2 vs 4.4 +/- 1.9 points, p <0.001). CONCLUSIONS: Results suggest that an increasing number of systematic reviews are published in the urological literature. However, many systematic reviews fail to meet established methodological standards, raising concerns about validity. Increased efforts are indicated to promote quality standards for performing systematic reviews among the authors and readership of the urological literature.