Key result
Large language models favor neuraxial over general anesthesia and recommend nerve blocks in hip fracture vignettes.
Why the study?
LLMs show promise for perioperative decision support, but persistent issues like hallucinations, miscalibration, and biases require rigorous evaluation before clinical use to determine how patient and surgical factors affect performance.
Do large language models provide appropriate anesthesia recommendations for hip fracture surgery vignettes?
Do large language models provide appropriate anesthesia recommendations for hip fracture surgery vignettes?
While LLMs provide generally reasonable anesthesia recommendations for hip fracture surgery, their systematic preferences diverge from contemporary evidence, highlighting the need for critical evaluation before clinical use.
LLM perioperative recommendations warrant caution before use; leaves open reliability across models and clinical scenarios.
BACKGROUND: Large language models (LLMs) show promise for perioperative decision support, but persistent issues, including hallucinations, miscalibration, and biases, indicate they require rigorous evaluation before clinical use. As LLM adoption increases in perioperative settings, systematic evaluation is needed to determine how patient and surgical factors affect performance. METHODS: We evaluated five general-purpose LLMs (DeepSeek 3.2, Gemini 2.5 Flash, GPT-5, GPT-5 mini, GPT-5 nano) using 216 standardized hip fracture surgery vignettes crossing six surgery types, two sexes, and 18 patient variables. We generated 50 samples per combination for 54,000 total responses and collected both structured recommendations and free-text justifications. We used logistic regression to estimate effects on three primary outcomes: anesthesia type, peripheral nerve block placement, and arterial line placement. In a limited sensitivity analysis, we evaluated two clinical LLMs (OpenEvidence, Doximity GPT) with 36 responses each. RESULTS: All models favored neuraxial over general anesthesia (76.1%–88.6% of responses), and all but DeepSeek 3.2 appropriately adjusted recommendations for relevant medical contraindications. All models except GPT-5 nano recommended preoperative peripheral nerve blocks (92.6%–99.3%) and were appropriately conservative regarding arterial line placement. However, free-text justifications frequently cited neuraxial benefits unsupported by recent randomized trials, and most models issued strong neuraxial recommendations despite a lack of clinical justification. We identified limited sociodemographic biases, with only one significant and clinically meaningful effect across 150 comparisons. Clinical LLMs provided similar recommendations to general-purpose models. CONCLUSIONS: While LLMs provided generally reasonable recommendations, systematic preferences diverging from contemporary evidence suggest uncritical use could shift practice patterns without improving patient outcomes.
No takes yet. Share an insight, caveat, or question.
Chen et al. (2026) studied Hip fracture surgery (n=54,000). Large Language Models (DeepSeek 3.2, Gemini 2.5 Flash, GPT-5, GPT-5 mini, GPT-5 nano) was evaluated on Anesthesia type, peripheral nerve block placement, and arterial line placement. Five large language models favored neuraxial over general anesthesia (76.1%-88.6%) and recommended preoperative peripheral nerve blocks (92.6%-99.3%) for hip fracture surgery vignettes.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: