Does the PRISM scoring system accurately predict the probability of mortality in an Italian pediatric ICU sample?
The PRISM scoring system showed good discrimination but poor calibration in an Italian pediatric ICU cohort, suggesting caution when using it to assess quality of care outside its original development context.
OBJECTIVE: To assess the validity of the Pediatric Risk of Mortality (PRISM) scoring system in accurately predicting the probability of mortality in an Italian intensive care unit (ICU) sample. DESIGN: Prospective, observational, multicenter study. SETTING: Twenty-six Italian ICUs classified into two groups: a) ICUs specifically dedicated to treating pediatric patients; and b) adult ICUs treating children on a regular basis. PATIENTS: Consecutive patients (n = 1,533) <15 yrs of age admitted during 1 yr. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: To assess the performance of the PRISM scoring system, the discrimination and calibration measures were adopted both in the whole population and in 12 preselected subgroups. A good discrimination capability of the scoring system was observed for both the whole population and subgroups (areas under the receiver operating characteristic curves were never <0.82). On the other hand, we documented an unsatisfactory calibration capability in the whole population and in most subgroups (p values of the Hosmer-Lemeshow goodness-of-fit test were <.001 in all but two subgroups). CONCLUSIONS: The analyses suggest that the unsatisfactory calibration of PRISM can be attributed to various reasons. Among those reasons, a poor performance of the system, as well as its sensitivity to factors not connected to clinical ICU performance, seem particularly important. A special caution is needed in adopting a severity of illness scoring system to assess quality of care, particularly in contexts different from the one in which the instrument was originally developed.
Bertolini et al. (Sat,) studied this question.