Accuracy and hallucinations in AI-generated medication dosing calculations: a sequential explanatory mixed-methods evaluation

Large language models (LLMs) are increasingly used in healthcare education and clinical decision support. However, their reliability for precise numerical tasks, particularly medication dosage calculations, remains inadequately evaluated. To evaluate the accuracy, consistency, and reasoning patterns of three LLMs: ChatGPT, Gemini, and DeepSeek, in performing standardized medication dosing calculations. A sequential explanatory mixed methods design was employed. Ninety standardized prompts covering basic, moderate, and advanced dosing scenarios were developed by subject matter experts and administered to each model, yielding 270 responses. Two independent clinical reviewers assessed responses using a structured checklist for numerical accuracy (correct ≤ 5%, minor deviation 5–10%, major deviation > 10%) and hallucination type. Descriptive statistics and Fisher’s exact tests were used for quantitative analysis, followed by directed content analysis of identified reasoning patterns. Interrater reliability was strong (overall agreement 91.3%; Cohen’s κ = 0.75–0.87). Gemini and DeepSeek achieved 100% accuracy across all complexity levels with no hallucinations. ChatGPT demonstrated an overall accuracy of 83.3%, with performance declining as complexity increased (53.3% correct on advanced items). Hallucinations occurred exclusively in ChatGPT and included incorrect calculation methods (16.7%), arithmetic errors (15.6%), and flawed reasoning chains (4.4%). Qualitative analysis revealed two archetypes: procedural transparency (Gemini/DeepSeek) and computational fragility (ChatGPT), characterised by methodological drift under increasing cognitive load. Significant inter-model differences exist in the accuracy and reasoning integrity of AI-generated medication dosing calculations. All AI-assisted calculations should be independently verified by qualified clinicians, particularly for high-alert medications and complex dosing protocols, before clinical application.