Comparison of Two Auto-Contouring Systems for Head and Neck Organs at Risk to Institutional Reference Standard in Radiotherapy Planning

Background: Manually contouring the organs at risk (OARs) is a time-consuming process with significant variability. Multiple commercial options are available to streamline this process by utilizing artificial intelligence. This study compares the performance of two commercially available systems, MIM Contour ProtégéAI v4.0.0 (MIM) and Radformation/Limbus AI v1.7.0 (Limbus), in auto-contouring head and neck (H&N) OARs. Methods: We conducted a retrospective single-institution study with 20 patients who received curative-intent H&N radiation therapy. Seventeen OARs, bilateral where applicable, were manually contoured for each patient, which established a reference for comparison. Dice coefficient (DC), mean Hausdorff distance (HDmean), and maximum Hausdorff distance (HDmax) were compared relative to the manual reference. Subjective contour quality was graded on a Likert scale. Wilcoxon signed-rank test with Benjamini–Hochberg false discovery rate procedures were used to compare outcomes. Results: Twenty patients were selected with a variety of H&N primary sites. The total number of OARs used for comparison was 480. Three quarters of patients were male, the mean age was 60.5 years (SD = 12.8), and the mean BMI was 23.53 kg/m2 (range 16.9–42.4). MIM performed more consistently with the manual reference (adjusted p < 0.05) compared to Limbus when contouring the bilateral brachial plexus and mandible. Limbus performed more consistently than MIM when contouring the left lacrimal gland, larynx, optic chiasm, bilateral optic nerves, right parotid, and esophagus. The DC, HDmean, and HDmax were not statistically different between MIM and Limbus for the remaining structures. Both AI algorithms had high performance on the mandible, eyes, thyroid, and salivary glands, and poor performance on the optic chiasm, lacrimal glands, and brachial plexuses. The subjective quality of the auto-contours from both systems were good, with most structures requiring no or minor changes only. The structures most likely to require revisions were the pharyngeal constrictors, larynx, and submandibular glands. Limbus achieved better subjective scores than MIM for four of seventeen structures. Conclusions: This comparison of two AI auto-contouring systems demonstrated performance consistent with the manual reference for most H&N OARs. Each system had strengths and weaknesses in auto-contouring specific OARs.