Answers are grounded in this paper and cite their sources.
BACKGROUND Accurate submission of medication claims to insurers has become increasingly complex in routine clinical practice. Conventional rule-based review systems are difficult to update and have limited utility in complex clinical settings. OBJECTIVE This study aimed to develop and validate a large language model (LLM)–based auditing system designed for on-premises deployment and integrated with the electronic health record (EHR). The system provides clinicians with reimbursement recommendations and the supporting rationale, offering a scalable and efficient solution for hospitals with limited clinical pharmacist capacity. METHODS Using real-world EHR data from 662 inpatients treated with 10 antineoplastic drugs, we compared four state-of-the-art LLMs that differed in architecture, parameter scale, and openness (Qwen3-4B, Qwen3-235B, DeepSeek-R1, and GPT-5) on a binary classification task of medical insurance reimbursement eligibility. We then developed an iterative prompt strategy and applied supervised fine-tuning (SFT) with activation-aware weight quantization (AWQ) to 4-bit precision for the lightweight model. Performance was assessed in three dimensions: predictive validity (accuracy, precision, specificity, and F1 score), repeat-test agreement (Cohen κ), and per-case inference latency. RESULTS With structured prompts, the three large models all exceeded 80% accuracy. Under the same prompt version (Prompt 2), GPT-5 performed best (accuracy 91.4%, Cohen κ = 0.97), followed by DeepSeek-R1 (88.1%, κ = 0.82) and Qwen3-235B (85.5%, κ = 0.69). After SFT and AWQ 4-bit quantization, Qwen3-4B improved from 54.5% to 80.1% accuracy and from 35.5% to 78.5% F1 score, while mean latency per case decreased from 11.5 s to 3.8 s. Trimming input tokens reduced the average token count per case by 83.9%. CONCLUSIONS LLM-based systems can achieve high accuracy in pre-auditing antineoplastic drug reimbursement, and optimized lightweight models offer a favorable trade-off between predictive performance and deployment cost. This study provides a reproducible framework for the local clinical implementation of AI-driven pharmacy administration tools. Multicenter prospective studies are warranted to validate generalizability and real-world impact.
No takes yet. Share an insight, caveat, or question.
Li et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: