Human-in-the-loop annotation and review platform for dental panoramic radiographs: a comparative evaluation of multimodal large language models in tooth-level abnormality detection

Multimodal large language models (MLLMs) are increasingly explored for medical image interpretation, yet their capability for tooth-level abnormality detection on dental panoramic radiographs (OPG) remains largely unquantified. Moreover, reference-standard construction for such evaluations usually lacks a transparent human-in-the-loop review pipeline. We developed an interactive web-based review platform with multi-view visualization (original and annotated images), finding-by-finding auditing, and FDI tooth-number recommendation with accept/reject/manual confirmation, from which an expert-reviewed OPG reference standard of 233 cases, admitted by consensus verification of two senior oral clinicians (1,859 tooth–finding pairs across 1,600 annotated tooth positions; FDI tooth numbers and a closed 20-class disease vocabulary), was derived. On this reference standard, we compared four MLLMs — doubao_web, Qwen3-VL-32B-Instruct, Gemini-3.6-flash-medium and M365 Copilot (GPT 5.6) — on two tasks: tooth-level finding presence and tooth–finding association (classification). The four models were accessed through two arms, an API arm and a consumer web arm, which were not run under matched conditions; within-arm contrasts are the primary inference and cross-arm contrasts are descriptive. Precision, recall and F1 were computed with case-level bootstrap 95% confidence intervals; paired binary outcomes were tested with exact McNemar tests, re-estimated under case-level cluster-robust variance with Holm correction; per-case F1 with Wilcoxon signed-rank tests; and inter-model error independence with Cohen’s kappa. The 233-case evaluation set was drawn from a corpus compiled from an openly licensed public collection together with radiographs acquired at the Department of Stomatology, Yijishan Hospital of Wannan Medical College, with all patient-identifiable information removed before analysis; the study was approved by the institutional ethics committee. M365 Copilot GPT 5.6 showed the highest point estimates on both tasks (tooth F1 = 0.642, pair F1 = 0.485), followed by Gemini-3.6-flash-medium (0.610/0.456). Within the web arm, M365 significantly outperformed doubao_web (0.501/0.348; both \(p < 0.001\) ); within the API arm, Gemini significantly outperformed Qwen3-VL-32B-Instruct (0.537/0.340; both \(p < 0.001\) ). The cross-arm gap between M365 and Gemini was borderline in per-case tooth F1 ( p = 0.052) and remained significant in the tooth-level McNemar test under case-level cluster-robust variance ( p = 0.005; Holm-adjusted p = 0.010). The two strongest models were positively correlated ( \(κ\) = 0.316), while all other pairwise model \(κ\) values were negative ( \(-0.203\) to \(-0.007\) ), indicating largely independent error patterns. All models performed poorly on caries and periapical lesions (pair F1 < 0.1). Current MLLMs performed relatively better on high-contrast structures (e.g., impacted teeth, F1 = 0.831 for M365, 0.814 for Gemini-3.6) but remained inadequate for subtle lesions; the errors of the four models were largely independent rather than complementary, and a two-model cross-check improved tooth-level union recall from 0.668 to 0.727–0.854 at the cost of arbitrating 20.4–42.3% of positive teeth (a descriptive cross-model quantification, given the unmatched access arms), which supports a two-model cross-check plus human review workflow when that arbitration cost is acceptable. The proposed review platform provides a reproducible, audit-trail reference-standard construction pipeline for benchmarking MLLMs in dental imaging.