Validating and updating a prediction rule for neurological sequelae after childhood bacterial meningitis

Recently, a prediction rule for developing neurological sequelae after childhood bacterial meningitis was developed on a small derivation set. Before implementing in practice a prediction rule must first be tested in new patients (external validation). Our aim was to study the external validity of this rule and, if necessary, to update the rule. The prediction rule was tested on newly available data (validation set) by assessing the rule's calibration and discrimination. We updated the prediction rule by adding extra predictors and re-estimating the regression coefficients of the original predictors in the combined datasets. The rule showed poor agreement between predicted risks and observed frequencies. The ROC area was 0.65 (95% CI 0.57-0.72), which was statistically significantly lower than in the derivation set (0.87 (0.78-0.96)), p-value<0.01. The updated prediction rule showed adequate performance in the combined data sets; the ROC area was 0.77 (95% CI 0.72-0.82). Further study of the generalizability of this updated rule may stimulate application in clinical practice.