A computer vision model for automated kidney stone segmentation and evaluation of its performance vs surgeons

Objectives To develop a computer vision model that segments stones to improve visualisation during ureteroscopy (URS) and to compare model performance to that of experts. Materials and Methods We collected 136 videos of URS for intrarenal kidney stone treatment. Frames were extracted at 3 frames per second (FPS) and manually annotated. The video dataset was split into training (75%), validation (5%) and testing (20%) subsets. Model performance was evaluated for stone localisation, laser ablation, and final evaluation of remaining fragments based on area under the receiver‐operating curve, binary cross‐entropy loss and Dice similarity coefficient (DSC). Model performance was compared to the manual annotations of five board‐certified urologists through pairwise comparison of frame‐by‐frame segmentation accuracy. Results The final dataset consisted of 21 718 frames from 38 fibreoptic and 98 digital videos. Overall, the model showed excellent performance: DSC 0.97 (interquartile range IQR 0.91, 0.99) and could segment at 30 FPS. Performance was similar for both fibreoptic (0.97 IQR 0.91, 0.99) and digital scopes (0.97 IQR 0.92, 0.99). Additionally, the model demonstrated good performance during stone localisation (0.98 IQR 0.93, 0.99) and stone laser ablation (0.96 IQR 0.89, 0.97), with slightly worse performance during evaluation of residual fragments (0.91 IQR 0.50, 0.97). Model performance was comparable to the five expert surgeons overall. In a head‐to‐head comparison, the model significantly outperformed three of the five experts and performed similarly to the other two. Conclusion The computer vision model demonstrates good performance for task‐specific stone segmentation evaluation during URS. The segmentation performance of the model was similar to the segmentation performance of expert surgeons, demonstrating the feasibility of its real‐time intra‐operative utilisation.