Biomedical Text Classification Using Transformer Models for Early Autism Prediction

Autism Spectrum Disorder (ASD) is a neurodevelopmental condition marked by challenges in communication, behavior, and social interaction. Early and accurate detection is vital for timely intervention. This study proposes a novel biomedical text classification framework leveraging transformer-based language models to detect ASD traits from unstructured clinical and caregiver narratives. The dataset combines real-world narratives sourced from Mendeley with synthetic samples generated via TinyLLaMA, improving class balance and linguistic diversity. Multiple pre-trained models-BioClinicalBERT, BlueBERT, and BERT-baseuncased-were evaluated under three training regimes: frozen embeddings with Logistic Regression, standard fine-tuning, and parameter-efficient fine-tuning (PEFT) using Low-Rank Adaptation (LoRA). Fine-tuning BioClinicalBERT on the last six transformer layers achieved the best binary classification accuracy of 96.67%, surpassing previous baselines. Additionally, a supervised multi-class classification task aligned with Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) symptom categories attained a macro F1-score of 78%, offering symptom-specific granularity. Key innovations include synthetic data augmentation, label smoothing, and layer-wise learning rate decay, which enhance model generalization while reducing training costs. Explainable AI methods, particularly Local Interpretable Model-agnostic Explanations (LIME), provide token-level interpretability, fostering clinical trust. This dualpipeline framework-supporting both binary and multi-class classification-demonstrates robust, scalable, and interpretable ASD detection capabilities. It offers practical utility for telehealth and resource-constrained clinical environments, paving the way for improved early screening and personalized intervention planning.