Fine-tuned large language models for early identification of autism spectrum disorder: a multicenter study in China

Early identification of autism spectrum disorder (ASD) and its differentiation from other neurodevelopmental disorders are crucial for timely interventions. However, overlapping symptoms among neurodevelopmental disorders complicate early diagnosis. This study aims to evaluate the potential of fine-tuned large language models as a diagnostic aid for early identification of ASD through analysis of electronic medical records. A multicenter cross-sectional study and a single-center retrospective cohort study were conducted between January 2019 and September 2024. The cross-sectional study included 3,940 children from Chongqing for model training with 10-fold validation and 1,657 children from nine other Chinese cities for external validation. Participants were aged 1.5–6 years with ASD, global developmental delay (GDD), or language delay (LD). The retrospective cohort comprised 584 children initially diagnosed with GDD/LD but later confirmed with ASD. ChatGLM4-9B and LLaMA3-8B were fine-tuned using low-rank adaptation to develop GLM4-ASD and LLaMA3-ASD. Model performance was compared with baseline, commercial (GPT-4, ERNIE−4.0), and healthcare-specific (Huatuo2-7B) models. In internal validation, GLM4-ASD showed the best performance (AUC 0.839, accuracy 83.7%, sensitivity 82.6%, specificity 85.2%, F1-score 84.8%), followed by LLaMA3-ASD, GPT-4, ERNIE−4.0, and Huatuo2-7B. In external validation, GLM4-ASD maintained robust performance (AUC 0.873, accuracy 87.9%), particularly in children under 3 years (AUC 0.882, accuracy 90.5%). In the longitudinal cohort, GLM4-ASD identified ASD earlier than pediatricians in 44.2% of cases (mean age 3.2 years), aligned in 40.1%, and delayed or missed diagnosis in 15.8% (mean age 4.7 years). The fine-tuned GLM4-ASD model shows promise as a decision-support system for assisting pediatricians in early ASD identification from medical records, especially in children under 3 years. However, as a retrospective study, the current findings primarily reflect classification performance rather than real-world diagnostic benefit. Future prospective studies are needed to further validate its clinical utility.