Reliability and Predictive Validity of a Consensus-Based Risk Assessment Tool

This study examines the reliability and predictive validity of the risk assessment tool used in Ontario, Canada. For the reliability portion of the study, a stratified random sample of 132 cases receiving services was drawn from one of Ontario's largest children's aid societies. Predictive validity was tested on 1,118 cases that were selected and administratively followed for varying lengths of time. Internal consistency was poor to fair; inter-rater reliability was greater than would be expected by chance alone in eight of the 23 risk items. Survival analysis revealed mostly poor predictive capacity for individual items and no predictive capacity for caseworkers' subjective overall risk rating.