Correctness and Completeness of a Multimodal Large Language Model in the Diagnosis and Treatment Recommendations for Proximal Femoral Fractures

Background and Objectives: To compare the diagnostic and therapeutic performance of a multimodal large language model (GPT) with that of two human evaluators in proximal femoral fracture (PFF) assessment, focusing on correctness and completeness. Materials and Methods: A consecutive series of 250 PFFs was reconstructed into standardized clinical scenarios including radiographs and structured clinical data. Each case was independently evaluated by a trauma fellow, a second-year orthopaedic resident, and GPT. Responses were scored for diagnostic and treatment correctness and completeness. Severe treatment errors were predefined as category-changing mismatches, failure to recognize an existing implant, or incorrect laterality. Results: Fully correct diagnoses were achieved in 97.2% of cases by the fellow, 44.8% by the resident, and 14.0% by GPT (p < 0.001). GPT showed lower odds of correct diagnosis compared with the resident (matched OR 0.23, 95% CI 0.14–0.37) and fellow (OR 0.005, 95% CI 0.0001–0.03), with clustering of errors in extracapsular fractures. Diagnostic completeness also differed significantly (p < 0.001), with GPT providing fully complete assessments in 60.4% of the cases (151/250). Fully correct treatment recommendations were provided in 98.8%, 81.2%, and 36.4% of cases, respectively (p < 0.001). Severe treatment errors occurred in 58.8% of GPT outputs versus 2.0% and 0% for the resident and fellow (OR vs. resident 36.5, 95% CI 13.95–135.75). Although treatment completeness was high (93.2%), 58.8% (147/250) of GPT responses were complete but incorrect. Conclusions: GPT demonstrated substantial limitations in both diagnostic and therapeutic reasoning for PFF, with frequent pattern-dependent errors and a high rate of clinically consequential treatment mismatches.