Objective: To externally validate a set of 9 ICD-10-CM billing diagnosis codes to identify pediatric ED encounters with agitation and aggression by determining their sensitivity and specificity compared with electronic health record (EHR) review. Methods: We performed a retrospective cross-sectional single-test diagnostic accuracy study of mental health encounters in the pediatric ED of an academic urban children’s hospital, utilizing EHR data for mental health encounters by children 6 to 18 years old from April 12, 2023, to December 30, 2023. The index test was the presence of any of 9 aggression-related diagnosis codes as utilized by Peleggi et al The reference test consisted of a structured EHR review to confirm that agitation/aggression was a reason for the visit. Sensitivity, specificity, positive predictive value and negative predictive value were calculated with 95% CIs. Results: We identified 855 pediatric mental health encounters (63%, 13 to 18 y old, 59% female). The diagnosis code set identified agitation/aggression in 62 encounters (7%), compared with 118 encounters (14%) confirmed to have agitation/aggression based on clinician notes. The 9 diagnosis codes yielded a sensitivity of 40% (95% CI: 31%, 49%) and specificity of 98% (95% CI: 97%, 99%) compared with EHR review, with a positive predictive value of 76% (63%, 85%) and a negative predictive value of 91% (89%, 93%). Conclusions: Upon external validation, a set of 9 billing diagnosis codes had poor sensitivity and excellent specificity for the identification of pediatric ED visits with agitation/aggression. Novel methods such as natural language processing may be needed to accurately identify pediatric ED visits with aggression/agitation.
Wnorowska et al. (Mon,) studied this question.