Association of skill and errors with outcomes in robotic rectal cancer surgery

Abstract Background Proficiency-based progression is key to analyzing and improving surgical performance. Objective assessment has demonstrated a direct link between operative performance and outcomes in laparoscopic surgery but not in robotics. There is current research to automate assessment processes with sensor data and machine learning. This requires granular, reliable annotations to train clinically implementable, trusted models, to improve patient safety. Aim To evaluate objective skill and error tools in robotic rectal cancer surgery, to provide a granular validated dataset from which to train and test deep learning models. Methodology A national, ethically approved, multicentre study, Video Analysis in Minimally Invasive Surgery (VAMIS) (ClinicalTrials.gov NCT05279287), recorded robotic-assisted total mesorectal excision (RTME). Recruited participants were pseudonymised and clinical data were collected. Operations were recorded and uploaded to Touch Surgery™ using the DS1 computer (Digital Technologies, a Medtronic company) and annotated by independent, blinded raters. Objective assessment employed error, Objective Clinical Human Reliability Analysis (OCHRA), Modifiable-Global Evaluative Assessment in Robotic Skills (M-GEARS) and TME performance tools. Correlational and multivariable regression analyses were performed, investigating associations between intraoperative skill and errors with clinical outcomes. Results 30 RTME operations were recorded, annotating 538 errors (median 13/operation). Major consequential errors were significantly associated with complications ( p = 0.031). Weighted error variables, accounting for error severity, were significantly associated with increased odds of prolonged operative time ( p = 0.025). Inter-rater reliability demonstrated an excellent matched error agreement percentage of two raters (mean agreement 90% (range 68–100%), after calibration sessions). OCHRA was significantly correlated with M-GEARS ( r = − 0.54 to − 0.77, p < 0.001–0.002) and the RTME performance tool ( r = 0.74, p = 0.007). Conclusion This feasibility study validated the concept that granular error and skill annotations can be objectively measured and associated with clinical outcomes in robotic rectal cancer surgery. This is an important step for larger studies and in aiding the development of deep learning models to predict errors and skill.