Using artificial intelligence for clinical trial quality assurance in a multi-institutional neurovascular-sparing prostate radiotherapy trial

Our study evaluated whether a deep learning auto-segmentation model combined with machine learning triage could support radiotherapy clinical trial quality assurance (QA). We analyzed 107 stereotactic ablative radiotherapy (SABR) cases from a multi-institutional phase II clinical trial of neurovascular sparing prostate SABR, focusing on physician contours of the internal pudendal artery (IPA) as a novel organ-at-risk with substantial interobserver variability. Contours were scored by the trial principal investigator as Per-Protocol ( n = 89) or Minor Deviation/Unacceptable ( n = 18). We applied a deep learning model for IPA auto-segmentation. Agreement between human and AI contours was then quantified using 14 overlap, distance, and surface metrics, and a supervised classifier was trained on these metrics to flag clinical trial protocol deviations. While AI segmentation achieved only modest geometric agreement with mean Dice similarity coefficient of 0.446 and 95th percentile Hausdorff distance of 14.23, when incorporating all 14 metrics, a machine learning classifier yielded an AUROC of 0.836 (95% CI, 0.552–0.949), flagging all Minor Deviation/Unacceptable cases with 100% sensitivity (95% CI, 56.6–100%) on the 27 case hold-out set with 6 false positives and no false negatives. AI segmentation combined with metrics-based machine learning can triage protocol deviations within a multi-institutional radiotherapy clinical trial, supporting prospective evaluation of AI-assisted trial QA. However, the small number of hold-out protocol deviation cases in this analysis limits generalizability, and the clinical acceptability of the AI contours and potential reductions in QA workload require further investigation.