Diagnostic agreement of fully automated AI-based and manual cephalometric tracing with an expert Photoshop reference: a paired method-comparison study

Purpose Manual cephalometric tracing is operator-dependent, and paired validation of fully automated artificial-intelligence (AI) platforms against an expert reference remains limited. We quantified the agreement of a fully automated AI-based cephalometric analysis platform and manual tracing with an expert digital reference.Materials and methods Retrospective, single-centre method-comparison study. One hundred cephalograms were traced by expert Photoshop (Reference), a fully automated AI-based platform, and manual acetate. Nineteen measurements and three classifications (skeletal class, vertical pattern, maxillary position) were obtained per method. Agreement was evaluated with Friedman tests, Bland–Altman bias and 95% limits of agreement (LoA) with confidence intervals, proportion within ±2°/±2 mm (McNemar), and squared-weighted kappa.Results On the angular measurements that drive skeletal, vertical, and dentoalveolar classification, AI-vs-Reference bias was small and 95% limits of agreement were narrow, whereas Manual-vs-Reference bias was several-fold larger with markedly wider limits. AI exceeded Manual in the proportion of cases within an a priori clinical tolerance on 17 of 19 measurements (significantly on 14 by paired McNemar, whereas Co–Gn significantly favoured Manual). AI-vs-Reference agreement on the three derived clinical classifications was substantial-to-almost-perfect (weighted κ = 0.73–0.85), whereas Manual-vs-Reference agreement was only fair (weighted κ = 0.28–0.33). Both methods performed poorly on the long linear distances (Effective Maxillary Length, Effective Mandibular Length, Anterior Lower Face Height).Conclusion In this single-operator, single-centre cohort, the AI-based platform agreed substantially with the expert reference on routinely used measurements and outperformed manual tracing. Operator-supervised verification of Condylion, ANS, and Menton remains necessary before AI long-linear values inform surgical planning.