What question did this study set out to answer?

The study aims to quantify interobserver variability in ultrasound imaging of parotid tumors and model decision patterns using machine learning.

March 18, 2026Open Access

Differential Diagnosis of Parotid Tumors on Ultrasound: Interobserver Variability and Examiner-Specific Decision Rules—A Machine Learning Approach

Key Points

The study aims to quantify interobserver variability in ultrasound imaging of parotid tumors and model decision patterns using machine learning.
Retrospective analysis of ultrasound images of 149 parotid tumors
Six examiners rated images using predefined descriptors
Accuracy and area under the curve (AUC) were calculated for each examiner
Interobserver agreement assessed with Cohen’s and Fleiss’ κ
Decision-tree models were trained on descriptors and metadata to visualize decision pathways.
Examiner accuracy ranged from 63.5% to 90.5%, AUC from 0.66 to 0.89
Objective descriptors showed higher agreement (size: κ = 0.57–0.97) than subjective descriptors (echogenicity: κ = 0.11–0.79)
Decision-tree accuracy varied: 57.2% to 80.0% (unpruned) and 65.1% to 76.5% (pruned)
High coverage for decision trees was noted (95.3–98.7%)
Tree complexity did not correlate consistently with examiner performance.

Abstract

Background/Objectives: Noninvasive differentiation of parotid gland tumors remains challenging despite ultrasound being the primary imaging modality for salivary gland lesions. Given its examiner dependence, improving diagnostic consistency and transparency is crucial. We quantified interobserver variability in parotid ultrasound, modeled examiner-specific decision patterns using machine learning surrogates, and tested whether surrogate complexity relates to examiner performance. Methods: In this retrospective, single-center study, six examiners independently rated ultrasound images of 149 parotid tumors using predefined descriptors. Performance was summarized using accuracy and the area under the receiver operating characteristic curve (AUC), with 95% confidence intervals (CIs). AUCs were compared using DeLong tests (Holm-adjusted). Interobserver agreement was assessed using pairwise Cohen’s and global Fleiss’ κ. For each examiner, a decision-tree surrogate was trained from structured descriptors and clinical metadata to reproduce examiner labels and visualize decision pathways; performance was estimated by 5-fold cross-validation. Results: Examiner accuracy ranged from 63.5% to 90.5% and AUC from 0.66 to 0.89 (best 0.89, 95% CI 0.83–0.95); the best performer exceeded the two lowest performers (p < 0.001). Agreement was higher for objective descriptors (size: κ = 0.57–0.97) than for subjective descriptors (echogenicity: κ = 0.11–0.79). Surrogate decision-tree accuracy versus histopathology ranged from 57.2% to 80.0% for unpruned and from 65.1% to 76.5% for pruned models, with high coverage (95.3–98.7%). Tree complexity showed no consistent association with examiner performance. Conclusions: Parotid ultrasound shows substantial interobserver variability. Interpretable surrogates can approximate individual labeling behavior from structured descriptors and clinical metadata, making examiner-dependent decision patterns explicit.

Read Full Paperexternally

Bookmark

View Full Paper

Cite This Study

Pillong et al. (Mon,) studied this question.

synapsesocial.com/papers/69ba42ee4e9516ffd37a3b1a https://doi.org/https://doi.org/10.3390/diagnostics16060880

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

Bookmark

View Full Paper