External Validation of the Risk Assessment of Morbidity in Pediatric Surgery Score

BACKGROUND: The Risk Assessment of Morbidity in Pediatric Surgery (RAMPS) score was developed by a tertiary care children’s center to predict risk of noncardiac pediatric surgical morbidity using intrinsic surgical risk (ISR) and individual comorbidity. The score demonstrated good internal validity, discrimination, and generalizability. This study seeks to externally validate the score in a similar patient population at a different tertiary care children’s hospital. METHODS: Following institutional review board (IRB) approval, data for all noncardiac, nontrauma, and nontransplant pediatric surgeries performed at the external children’s hospital in 2019 were collected via review of medical records. The RAMPS score for each patient was calculated. A composite morbidity score defined as transfer to intensive care unit (ICU), cardiac arrest, unplanned endotracheal intubation, or unplanned initiation of positive pressure ventilation (PPV) in the 72 hours following surgery was calculated. Discrimination was evaluated using receiver operating characteristic curve analysis, and calibration between predicted and observed risk of morbidity was assessed. RESULTS: Of 13,718 patients included, 409 (2.98%) experienced the composite morbidity outcome. The most common morbidity was postoperative PPV—361 (2.63%). Only 1.5% (n = 205) of the cohort was critically ill, though 14.1% (n = 1934) had at least three chronic conditions and 19.5% (n = 2668) of patients had a procedure in ISR quartiles 3 or 4. Each component of the RAMPS score was significantly associated with the outcome. The RAMPS score had very good discrimination (area under the receiver operating characteristic curve [AUC] = 0.829; 95% confidence interval [CI], 0.809–0.849) and good calibration between observed and predicted probabilities (Brier score = 0.03). CONCLUSIONS: The RAMPS score performed well in this external validation with overall less patient complexity and less morbidity than in the original cohort included for development and internal validation. The score moderately overestimated risk in the most complex patients, which does not diminish its utility in guiding resource allocation to manage or prevent perioperative adverse events. In summary, the RAMPs score is a simple, highly discriminant, well-calibrated, and externally validated tool clinicians and health systems can use to predict morbidity among pediatric surgery patients.