Automating Injury Severity Score Calculation Using Large Language Models: A Feasibility Study With Large Language Model–Assisted Trauma Scoring

INTRODUCTION: The injury severity score (ISS) is a crucial tool for trauma severity assessment, but its calculation relies on registrars manually assigning the scores, a process prone to human error and time constraints. This study evaluates the feasibility of using a large language model (LLM) to assist in ISS calculation based on trauma patients' diagnoses. METHODS: A retrospective study was conducted at a level I trauma center. Training data from trauma patients hospitalized in 2022 were used, and we retrieved the final diagnoses, abbreviated injury scale scores, and ISS values assigned by experienced registrars. The LLM was trained with structured prompts detailing trauma scoring principles. The model was validated using 100 randomly selected trauma cases from 2022, comparing LLM-generated ISS (LLM ISS) with registrar-calculated ISS. The correlation was evaluated using Pearson correlation, and the agreement was evaluated using the intraclass correlation coefficient (ICC) and Bland-Altman analysis. RESULTS: Among the 100 trauma patients, the ISS distribution showed 34 patients with ISS <9, 33 with ISS 9-16, and 33 with ISS >16. The intraclass correlation coefficient between LLM ISS and registrar-calculated ISS was 0.981 (95% confidence interval: 0.97, 0.99), with an accuracy of 0.91. Bland-Altman analysis showed a mean bias of -0.03 indicating strong consistency. CONCLUSIONS: LLM ISS demonstrated high reliability and accuracy, offering a promising approach to automate trauma scoring. Future research should explore its integration into real-time clinical workflows and its expansion to other trauma severity scores.