C25-16 The Next Step in Predicting Asthma Exacerbations - A Large Language Model for Extracting Pediatric Asthma Symptoms and Triggers

Rationale Pediatric asthma is one of the most prevalent chronic diseases, with symptoms commonly triggered by physical activity, respiratory infections, or allergies. The disease follows an episodic course with potential acute exacerbations. Identifying patterns in symptoms and triggers is important for anticipating these exacerbations. Electronic patient records (EPRs) contain valuable longitudinal information on children’s symptoms and triggers, but this information is often unstructured and difficult for healthcare professionals to interpret effectively. Artificial intelligence can handle language processing to extract this information automatically. However, no validated large language model (LLM) pipeline currently exists to analyze the combination of multiple asthma-related symptoms or triggers in Dutch EPRs. In this study, we aim to increase asthma exacerbation prediction by developing an LLM-based pipeline for detecting and classifying assertion of key asthma symptoms and triggers. Methods This study is part of the retrospective PREVENT study with 2000 pediatric asthma patients at Medisch Spectrum Twente hospital, the Netherlands. The dataset contains 6000 anamneses between 2021 and 2025. 500 anamneses were manually labeled by one trained clinical researcher, 10% revision by a second researcher, for presence or absence of clinical symptoms and triggers: dyspnea, wheezing, coughing, exercise-induced symptoms, nocturnal symptoms, smoking/vaping, and salbutamol use. The pipeline (Figure 1) consists of the existing Dutch LLMs XLM RoBERTa Large for detection and BERTje for assertion classification (absence or presence). We used 400 anamneses for training and 100 for evaluation. The LLM-based pipeline’s performance was assessed through accuracy and F1 score. This pipeline serves as an input for the asthma exacerbation prediction model. Results The overall LLM-based pipeline achieved 86% accuracy on pediatric symptoms/triggers detection and classification. The symptom detection achieved a micro F1-score of 0.91, and assertion classification a macro F1-score of 0.91. Performance was highest for “wheezing”, “fever”, and for “dyspnea” (97% accuracy). Performance was lower for “comorbidities” (40%) and “unspecified complaints” (42%), primarily due to vague label description. Conclusions Our developed LLM-based pipeline can identify key asthma symptoms and triggers from unstructured anamneses with high accuracy. This serves as an important step in using symptoms and triggers for predicting asthma exacerbations and determining the patient’s risk factors. Future work will focus on refining label definitions, expanding training data, and incorporating reinforcement learning in all anamneses. Ultimately, this prediction pipeline paves the way for a shift from reactive to proactive asthma care, enabling earlier intervention.