Testing the Cambridge Quality Checklists on a review of disrupted families and crime

BACKGROUND: Systematic reviews of the relationship between non-manipulated factors (e.g. low empathy) and offending are becoming more common, and it is important to consider the methodological quality of studies included in such reviews. AIMS: To assess aspects of the reliability and validity of the Cambridge Quality Checklists, a set of three measures for examining the methodological quality of studies included in systematic reviews of risk factors for offending. METHODS: All 60 studies in a systematic review of disrupted families and offending were coded on the CQC and codes compared with the effect sizes derived from the studies. RESULTS: Overall, the CQC was easy to score, and the relevant information was available in most studies. The scales had high inter-rater reliability. Only 13 studies scored high on the Checklist of Correlates, 18 scored highly on the Checklist of Risk Factors and none scored highly on the Checklist of Causal Risk Factors. Generally, studies that were of lower quality had higher effect sizes. CONCLUSIONS: The CQC could be a useful method of assessing the methodological quality of studies of risk factors for offending but might benefit from additional conceptual work, changes to the wording of some scales and additional levels for scoring.