Evaluating QLoRA Fine-tuning of Small Language Models for Structured Neurological Clinical Reasoning: A Comparative Study

Large language models (LLMs) have demonstrated strong potential for clinical decision support, yetmost existing research focuses on large proprietary models that are computationally expensive and diffi-cult to deploy in privacy-sensitive healthcare environments. This paper investigates whether parameter-efficient fine-tuning can enable structured neurological clinical reasoning in small, open-source languagemodels. We fine-tune Qwen2.5-0.5B-Instruct and Qwen2.5-3B-Instruct using QLoRA on a 500-exampleneurology dataset covering four clinical categories (stroke, headaches, seizures, and parkinsonism). Theresulting models are compared against their respective base models on 40 previously unseen clinicalcases evaluated under both narrative and structured prompt formats to assess reasoning quality andprompt robustness. Our results show that model scale has a substantially greater impact on clinicalreasoning performance than parameter-efficient fine-tuning alone. The 3B model consistently achieveshigher training accuracy, exhibits more stable optimization, and produces more reliable neuroanatomi-cal localization and management recommendations than the fine-tuned 0.5B model, even without task-specific fine-tuning. In contrast, QLoRA fine-tuning substantially improves the smaller model by reduc-ing hallucinations and increasing the structural completeness and consistency of its clinical responsesacross both prompt formats. We also observe a decrease in differential-diagnosis completeness for thefine-tuned 3B model under the prompt format most closely matching its training distribution. How-ever, bootstrap confidence intervals overlap substantially, indicating that this trend should be interpretedas suggestive rather than statistically conclusive. We hypothesize that structural inconsistencies withinthe training data may have contributed to this behavior, motivating future investigation using larger andmore standardized datasets. To support reproducibility and further research on efficient clinical languagemodels, we release our training methodology, dataset construction pipeline, evaluation framework, andcomplete experimental results.