Introduction: The Functional Status Scale (FSS) is commonly used to assess morbidity among critically ill children, but manual score assignment is time-intensive. We developed and evaluated a fine-tuned version of OpenAI’s generative pretrained transformer (GPT)-4o large language model (hereafter FSS-AI) to estimate FSS scores from clinical notes at key hospitalization timepoints. Methods: We conducted a secondary analysis of a prospective cohort of children (1 month–18 years) who required mechanical ventilation for ≥3 days at a quaternary academic children’s hospital. FSS-AI was fine-tuned using clinical notes (history and physical, transfer summary, and discharge summary) and manually assigned FSS scores from 11 patients to estimate FSS at three timepoints: baseline (pre-illness), PICU transfer, and hospital discharge. The model was then applied to 428 unseen notes. Agreement with gold standard scores—assigned by the research team during the original study—was evaluated using weighted Cohen’s Kappa. Discriminative performance for identifying normal (FSS 6–7) vs. abnormal (FSS ≥8) status and new morbidity at hospital discharge (FSS increase ≥3 or domain increase ≥2) was also evaluated. Results: FSS-AI analyzed 428 notes in 4 hours and 15 minutes (35.7 seconds per note). Agreement with gold standard FSS scores was moderate at baseline (Kappa 0.59, 95% confidence interval 0.49–0.70), PICU transfer (0.45, 0.37–0.54), and hospital discharge (0.51, 0.43–0.58). For classifying normal vs. abnormal FSS scores, accuracy and sensitivity were highest at the baseline (0.90 and 0.93, respectively) and hospital discharge (0.81 and 0.88) timepoints. Accuracy and sensitivity for identification of new morbidity at discharge were 0.75 and 0.56, respectively. Conclusions: A fine-tuned GPT-4o model was able to generate FSS scores from clinical notes for three hospitalization timepoints among critically ill children. The model showed moderate agreement with manually assigned scores (highest at the baseline and hospital discharge timepoints) and could accurately discriminate normal vs. abnormal functional status, supporting its potential use for automated functional outcome tracking in the PICU.
Martin et al. (Sun,) studied this question.