Multimodal large language model assistance in interpreting postoperative calcaneal radiographs: a multicenter multi-reader study

This is the authors' abstract. We don't add key points for this paper.

Multimodal large language models (LLMs) may support radiographic reporting, but evidence in complex postoperative musculoskeletal imaging is limited. We compared case-level risk classification between a fixed-order LLM-assisted structured reporting session and an earlier unaided session. This retrospective multicenter multi-reader study included 73 postoperative calcaneal fixation cases from three campuses in one hospital system. Six radiologists interpreted each case unaided and, after a washout of more than 4 weeks with rerandomized order, reviewed and finalized a fixed pre-generated GPT-5 checklist and draft report in the same interface. Session order was not counterbalanced. Postoperative CT expert consensus was the pragmatic reference standard. The primary endpoint was accuracy at a binary action threshold pooling medium- and high-risk cases as requiring escalation. Across 438 case-reader observations per session, binary accuracy increased from 313/438 (71.5%) in Session 1 to 351/438 (80.1%) in Session 2 (difference, 8.7% points; cluster-bootstrap 95% CI, 1.1–17.4; OR, 1.97; 95% CI, 1.37–2.84; P < 0.001). Sensitivity increased from 54.2% to 79.2%, while specificity decreased from 85.0% to 80.9%. At the exact-high threshold, sensitivity was 45.0% and 43.3%, respectively. Mean recorded duration was 378.6 versus 98.4 s (adjusted ratio, 0.257; 95% CI, 0.248–0.266), and mean Brier score was 0.246 versus 0.177 (difference, 0.069; 95% CI, 0.007–0.147). Session 2 was associated with higher binary accuracy and shorter recorded duration, with higher sensitivity but lower specificity; sensitivity did not improve at the exact-high threshold. Because the model-populated session was always second, the study cannot separate model-generated content and report-prefill effects from order or familiarity. Prospective randomized or counterbalanced validation across independent institutions is needed.